mlsanityData to 7 Oct 2026

Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlo

arXiv 1806.05490 ↗First posted 14 Jun 2018stat.ML cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Deep Gaussian Processes (DGPs) are hierarchical generalizations of Gaussian Processes that combine well calibrated uncertainty estimates with the high flexibility of multilayer models. One of the biggest challenges with these models is that exact inference is intractable. The current state-of-the-art inference method, Variational Inference (VI), employs a Gaussian approximation to the posterior distribution. This can be a potentially poor unimodal approximation of the generally multimodal posterior. In this work, we provide evidence for the non-Gaussian nature of the posterior and we apply the Stochastic Gradient Hamiltonian Monte Carlo method to generate samples. To efficiently optimize the hyperparameters, we introduce the Moving Window MCEM algorithm. This results in significantly better predictions at a lower computational cost than its VI counterpart. Thus our method establishes a new state-of-the-art for inference in DGPs.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 5; 2023: 7; 2024: 4; 2025: 3; 2026: 2.

32 papers.

Sort:

By area:

  1. 2021Causal Inference for Time series Analysis: Problems, Methods and Evaluationcited by 42
  2. 2023Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorialcited by 27
  3. 2022The Neural Process Family: Survey, Applications and Perspectivescited by 23
  4. 2021The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspectivecited by 21
  5. 2022Bayesian Interpolation with Deep Linear Networkscited by 21
  6. 2024Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inferencecited by 16
  7. 2021Deep Gaussian Processes: A Surveycited by 13
  8. 2021GPflux: A Library for Deep Gaussian Processescited by 12
  9. 2021Deep Gaussian Process Emulation using Stochastic Imputationcited by 11
  10. 2024Universal Functional Regression with Neural Operator Flowscited by 9
  11. 2021Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Featurescited by 8
  12. 2022Deep Variational Implicit Processescited by 8
20 more
  1. 2023A Unifying Perspective on Non-Stationary Kernels for Deeper Gaussian Processescited by 8
  2. 2023Free-Form Variational Inference for Gaussian Process State-Space Modelscited by 7
  3. 2023Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processescited by 6
  4. 2021A Sparse Expansion For Deep Gaussian Processescited by 5
  5. 2022Sparse Gaussian Process Hyperparameters: Optimize or Integrate?cited by 4
  6. 2024Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equationscited by 4
  7. 2025Diffusion Bridge Variational Inference for Deep Gaussian Processescited by 4
  8. 2021Convolutional Normalizing Flows for Deep Gaussian Processescited by 2
  9. 2021Probabilistic Metamodels for an Efficient Characterization of Complex Driving Scenarioscited by 2
  10. 2025Deep Gaussian Process-based Cost-Aware Batch Bayesian Optimization for Complex Materials Design Campaignscited by 2
  11. 2021Conditional Deep Gaussian Processes: empirical Bayes hyperdata learningcited by 1
  12. 2022Distributional Gaussian Processes Layers for Out-of-Distribution Detectioncited by 1
  13. 2023Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learningcited by 1
  14. 2024Adaptive RKHS Fourier Features for Compositional Gaussian Process Modelscited by 1
  15. 2025Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functionscited by 1
  16. 2026Onsager-Machlup Posterior Transport for Deep Gaussian Processescited by 1
  17. 2021Subset-of-Data Variational Inference for Deep Gaussian-Processes Regressioncited by 0
  18. 2023Linked Deep Gaussian Process Emulation for Model Networkscited by 0
  19. 2023Deep Transformed Gaussian Processescited by 0
  20. 2026How Deep Are Deep GPs, Really? A Sharp Threshold and a Non-Gaussian Limit for Compositional GPscited by 0