# Bayesian Optimization using Deep Gaussian Processes

arXiv: https://arxiv.org/abs/1905.03350 · HTML: https://mlsanity.com/paper/1905.03350/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2019.

## Built on by (most cited first, 22 of 22)

- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [SMT 2.0: A Surrogate Modeling Toolbox with a focus on Hierarchical and Mixed Variables Gaussian Processes](https://mlsanity.com/paper/2305.13998/) (2023)
- [Diffusion Model for Data-Driven Black-Box Optimization](https://mlsanity.com/paper/2403.13219/) (2024)
- [Conditioning Sparse Variational Gaussian Processes for Online Decision-making](https://mlsanity.com/paper/2110.15172/) (2021)
- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs](https://mlsanity.com/paper/2509.17998/) (2025)
- [Multi-fidelity Bayesian Optimization: A Review](https://mlsanity.com/paper/2311.13050/) (2023)
- [Neural Process for Black-Box Model Optimization Under Bayesian Framework](https://mlsanity.com/paper/2104.02487/) (2021)
- [Deep Gaussian Process Emulation using Stochastic Imputation](https://mlsanity.com/paper/2107.01590/) (2021)
- [Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes](https://mlsanity.com/paper/2210.07612/) (2022)
- [A Sparse Expansion For Deep Gaussian Processes](https://mlsanity.com/paper/2112.05888/) (2021)
- [Heuristic-free Optimization of Force-Controlled Robot Search Strategies in Stochastic Environments](https://mlsanity.com/paper/2207.07524/) (2022)
- [Pseudo-Bayesian Optimization](https://mlsanity.com/paper/2310.09766/) (2023)
- [Accelerating material discovery with a threshold-driven hybrid acquisition policy-based Bayesian optimization](https://mlsanity.com/paper/2311.09591/) (2023)
- [Relaxed Gaussian process interpolation: a goal-oriented approach to Bayesian optimization](https://mlsanity.com/paper/2206.03034/) (2022)
- [Gaussian Process regression over discrete probability measures: on the non-stationarity relation between Euclidean and Wasserstein Squared Exponential Kernels](https://mlsanity.com/paper/2212.01310/) (2022)
- [Deep Gaussian Process-based Cost-Aware Batch Bayesian Optimization for Complex Materials Design Campaigns](https://mlsanity.com/paper/2509.14408/) (2025)
- [Polynomial Chaos Expanded Gaussian Process](https://mlsanity.com/paper/2405.01052/) (2024)
- [Global Optimisation of Black-Box Functions with Generative Models in the Wasserstein Space](https://mlsanity.com/paper/2407.11917/) (2024)
- [Gearing Gaussian process modeling and sequential design towards stochastic simulators](https://mlsanity.com/paper/2412.07306/) (2024)
- [Gradient-enhanced deep Gaussian processes for multifidelity modelling](https://mlsanity.com/paper/2402.16059/) (2024)
- [tidyHEBO: Improving the Robustness of Heteroskedastic Evolutionary Bayesian Optimization](https://mlsanity.com/paper/2607.10669/) (2026)

## Abstract (arXiv)

Bayesian Optimization using Gaussian Processes is a popular approach to deal with the optimization of expensive black-box functions. However, because of the a priori on the stationarity of the covariance matrix of classic Gaussian Processes, this method may not be adapted for non-stationary functions involved in the optimization problem. To overcome this issue, a new Bayesian Optimization approach is proposed. It is based on Deep Gaussian Processes as surrogate models instead of classic Gaussian Processes. This modeling technique increases the power of representation to capture the non-stationarity by simply considering a functional composition of stationary Gaussian Processes, providing a multiple layer structure. This paper proposes a new algorithm for Global Optimization by coupling Deep Gaussian Processes and Bayesian Optimization. The specificities of this optimization method are discussed and highlighted with academic test cases. The performance of the proposed algorithm is assessed on analytical test cases and an aerospace design optimization problem and compared to the state-of-the-art stationary and non-stationary Bayesian Optimization approaches.