# Active Learning for Deep Gaussian Process Surrogates

arXiv: https://arxiv.org/abs/2012.08015 · HTML: https://mlsanity.com/paper/2012.08015/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2020.

## Built on by (most cited first, 30 of 38)

- [Bayesian Active Learning with Fully Bayesian Gaussian Processes](https://mlsanity.com/paper/2205.10186/) (2022)
- [Deep Gaussian Process Emulation using Stochastic Imputation](https://mlsanity.com/paper/2107.01590/) (2021)
- [Triangulation candidates for Bayesian optimization](https://mlsanity.com/paper/2112.07457/) (2021)
- [qPOTS: Efficient batch multiobjective Bayesian optimization via Pareto optimal Thompson sampling](https://mlsanity.com/paper/2310.15788/) (2023)
- [Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design](https://mlsanity.com/paper/2306.14348/) (2023)
- [A Sparse Expansion For Deep Gaussian Processes](https://mlsanity.com/paper/2112.05888/) (2021)
- [Active Learning of Piecewise Gaussian Process Surrogates](https://mlsanity.com/paper/2301.08789/) (2023)
- [Voronoi Candidates for Bayesian Optimization](https://mlsanity.com/paper/2402.04922/) (2024)
- [Sensitivity Prewarping for Local Surrogate Modeling](https://mlsanity.com/paper/2101.06296/) (2021)
- [Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems](https://mlsanity.com/paper/2303.10022/) (2023)
- [Bayesian Non-linear Latent Variable Modeling via Random Fourier Features](https://mlsanity.com/paper/2306.08352/) (2023)
- [All Emulators are Wrong, Many are Useful, and Some are More Useful Than Others: A Reproducible Comparison of Computer Model Surrogates](https://mlsanity.com/paper/2512.09060/) (2025)
- [Inverse Models for Estimating the Initial Condition of Spatio-Temporal Advection-Diffusion Processes](https://mlsanity.com/paper/2302.04134/) (2023)
- [Efficient optimization of expensive black-box simulators via marginal means, with application to neutrino detector design](https://mlsanity.com/paper/2508.01834/) (2025)
- [Informed Initialization for Bayesian Optimization and Active Learning](https://mlsanity.com/paper/2510.23681/) (2025)
- [Dynamic Bayesian Learning for Spatiotemporal Mechanistic Models](https://mlsanity.com/paper/2208.06528/) (2022)
- [Variational Bayesian surrogate modelling with application to robust design optimisation](https://mlsanity.com/paper/2404.14857/) (2024)
- [Polynomial Chaos Expanded Gaussian Process](https://mlsanity.com/paper/2405.01052/) (2024)
- [Future Aware Safe Active Learning of Time Varying Systems using Gaussian Processes](https://mlsanity.com/paper/2405.10581/) (2024)
- [Bayesian Adaptive Calibration and Optimal Design](https://mlsanity.com/paper/2405.14440/) (2024)
- [Local transfer learning Gaussian process modeling, with applications to surrogate modeling of expensive computer simulators](https://mlsanity.com/paper/2410.12690/) (2024)
- [Gearing Gaussian process modeling and sequential design towards stochastic simulators](https://mlsanity.com/paper/2412.07306/) (2024)
- [Sparse Techniques for Regression in Deep Gaussian Processes](https://mlsanity.com/paper/2505.11355/) (2025)
- [Constraint-Informed Active Learning for End-to-End ACOPF Optimization Proxies](https://mlsanity.com/paper/2511.06248/) (2025)
- [Mind the Jumps: A Scalable Robust Local Gaussian Process for Multidimensional Response Surfaces with Discontinuities](https://mlsanity.com/paper/2512.12574/) (2025)
- [Correcting Boundary Bias and Observation Independence in Bayesian Experimental Design](https://mlsanity.com/paper/2602.01898/) (2026)
- [Bayesian Emulation for Computer Models with Multiple Partial Discontinuities](https://mlsanity.com/paper/2210.10468/) (2022)
- [Active Discrimination Learning for Gaussian Process Models](https://mlsanity.com/paper/2211.11624/) (2022)
- [Bayesian Kernelized Tensor Factorization as Surrogate for Bayesian Optimization](https://mlsanity.com/paper/2302.14510/) (2023)
- [Linked Deep Gaussian Process Emulation for Model Networks](https://mlsanity.com/paper/2306.01212/) (2023)

## Abstract (arXiv)

Deep Gaussian processes (DGPs) are increasingly popular as predictive models in machine learning (ML) for their non-stationary flexibility and ability to cope with abrupt regime changes in training data. Here we explore DGPs as surrogates for computer simulation experiments whose response surfaces exhibit similar characteristics. In particular, we transport a DGP's automatic warping of the input space and full uncertainty quantification (UQ), via a novel elliptical slice sampling (ESS) Bayesian posterior inferential scheme, through to active learning (AL) strategies that distribute runs non-uniformly in the input space -- something an ordinary (stationary) GP could not do. Building up the design sequentially in this way allows smaller training sets, limiting both expensive evaluation of the simulator code and mitigating cubic costs of DGP inference. When training data sizes are kept small through careful acquisition, and with parsimonious layout of latent layers, the framework can be both effective and computationally tractable. Our methods are illustrated on simulation data and two real computer experiments of varying input dimensionality. We provide an open source implementation in the "deepgp" package on CRAN.