# Deep Gaussian Processes: A Survey

arXiv: https://arxiv.org/abs/2106.12135 · HTML: https://mlsanity.com/paper/2106.12135/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 21 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [Stochastic Variational Deep Kernel Learning](https://mlsanity.com/paper/1611.00336/) (2016)
- [Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1806.05490/) (2018)
- [Variational Auto-encoded Deep Gaussian Processes](https://mlsanity.com/paper/1511.06455/) (2015)
- [Energy-Based Processes for Exchangeable Data](https://mlsanity.com/paper/2003.07521/) (2020)

## Built on by (most cited first, 13 of 13)

- [Bayesian Learning for Neural Networks: an algorithmic survey](https://mlsanity.com/paper/2211.11865/) (2022)
- [Uncertainty Estimation for Molecules: Desiderata and Methods](https://mlsanity.com/paper/2306.14916/) (2023)
- [Multi-Task Neural Processes](https://mlsanity.com/paper/2111.05820/) (2021)
- [Gaussian Measures Conditioned on Nonlinear Observations: Consistency, MAP Estimators, and Simulation](https://mlsanity.com/paper/2405.13149/) (2024)
- [Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantification](https://mlsanity.com/paper/2511.10282/) (2025)
- [Knowledge-aware Evolutionary Graph Neural Architecture Search](https://mlsanity.com/paper/2411.17339/) (2024)
- [Relaxed Gaussian process interpolation: a goal-oriented approach to Bayesian optimization](https://mlsanity.com/paper/2206.03034/) (2022)
- [Uncertainty Disentanglement with Non-stationary Heteroscedastic Gaussian Processes for Active Learning](https://mlsanity.com/paper/2210.10964/) (2022)
- [Probabilistic Interactive 3D Segmentation with Hierarchical Neural Processes](https://mlsanity.com/paper/2505.01726/) (2025)
- [SGPMIL: Sparse Gaussian Process Multiple Instance Learning](https://mlsanity.com/paper/2507.08711/) (2025)
- [Deep Gaussian Processes for Air Quality Inference](https://mlsanity.com/paper/2211.10174/) (2022)
- [INR-Bench: A Unified Benchmark for Implicit Neural Representations in Multi-Domain Regression and Reconstruction](https://mlsanity.com/paper/2510.10188/) (2025)
- [MORE-PLR: multi-output regression employed for partial label ranking](https://mlsanity.com/paper/2609.29386/) (2026)

## Abstract (arXiv)

Gaussian processes are one of the dominant approaches in Bayesian learning. Although the approach has been applied to numerous problems with great success, it has a few fundamental limitations. Multiple methods in literature have addressed these limitations. However, there has not been a comprehensive survey of the topics as of yet. Most existing surveys focus on only one particular variant of Gaussian processes and their derivatives. This survey details the core motivations for using Gaussian processes, their mathematical formulations, limitations, and research themes that have flourished over the years to address said limitations. Furthermore, one particular research area is Deep Gaussian Processes (DGPs), it has improved substantially in the past decade. The significant publications that advanced the forefront of this research area are outlined in their survey. Finally, a brief discussion on open problems and research directions for future work is presented at the end.