# MCMC for Variationally Sparse Gaussian Processes

arXiv: https://arxiv.org/abs/1506.04000 · HTML: https://mlsanity.com/paper/1506.04000/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2015.

## Built on by (most cited first, 30 of 44)

- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2106.01121/) (2021)
- [Variational multiple shooting for Bayesian ODEs with Gaussian processes](https://mlsanity.com/paper/2106.10905/) (2021)
- [Safe Active Learning for Multi-Output Gaussian Processes](https://mlsanity.com/paper/2203.14849/) (2022)
- [SKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processes](https://mlsanity.com/paper/2106.06695/) (2021)
- [Bayesian Calibration of Imperfect Computer Models using Physics-Informed Priors](https://mlsanity.com/paper/2201.06463/) (2022)
- [Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes](https://mlsanity.com/paper/2302.04534/) (2023)
- [Improving sample efficiency of high dimensional Bayesian optimization with MCMC](https://mlsanity.com/paper/2401.02650/) (2024)
- [Variational Nearest Neighbor Gaussian Process](https://mlsanity.com/paper/2202.01694/) (2022)
- [Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processes](https://mlsanity.com/paper/2111.04558/) (2021)
- [Free-Form Variational Inference for Gaussian Process State-Space Models](https://mlsanity.com/paper/2302.09921/) (2023)
- [Hierarchical Inducing Point Gaussian Process for Inter-domain Observations](https://mlsanity.com/paper/2103.00393/) (2021)
- [Semi-Parametric Inducing Point Networks and Neural Processes](https://mlsanity.com/paper/2205.11718/) (2022)
- [Learning Energy Conserving Dynamics Efficiently with Hamiltonian Gaussian Processes](https://mlsanity.com/paper/2303.01925/) (2023)
- [Koopman-Equivariant Gaussian Processes](https://mlsanity.com/paper/2502.06645/) (2025)
- [Input Dependent Sparse Gaussian Processes](https://mlsanity.com/paper/2107.07281/) (2021)
- [Gaussian Processes for Missing Value Imputation](https://mlsanity.com/paper/2204.04648/) (2022)
- [Sparse Gaussian Process Hyperparameters: Optimize or Integrate?](https://mlsanity.com/paper/2211.02476/) (2022)
- [Actually Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2304.05091/) (2023)
- [Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equations](https://mlsanity.com/paper/2408.06069/) (2024)
- [Calibrated simplex-mapping classification](https://mlsanity.com/paper/2103.02926/) (2021)
- [Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling](https://mlsanity.com/paper/2505.16481/) (2025)
- [Transformer with Koopman-Enhanced Graph Convolutional Network for Spatiotemporal Dynamics Forecasting](https://mlsanity.com/paper/2507.03855/) (2025)
- [Gradient and Uncertainty Enhanced Sequential Sampling for Global Fit](https://mlsanity.com/paper/2310.00110/) (2023)
- [Recurrent Memory for Online Interdomain Gaussian Processes](https://mlsanity.com/paper/2502.08736/) (2025)
- [On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach](https://mlsanity.com/paper/2103.03321/) (2021)
- [Nonlinear Hawkes Process with Gaussian Process Self Effects](https://mlsanity.com/paper/2105.09618/) (2021)
- [Geometry-Aware Hierarchical Bayesian Learning on Manifolds](https://mlsanity.com/paper/2111.00184/) (2021)
- [Modelling calibration uncertainty in networks of environmental sensors](https://mlsanity.com/paper/2205.01988/) (2022)
- [Learning Physics between Digital Twins with Low-Fidelity Models and Physics-Informed Gaussian Processes](https://mlsanity.com/paper/2206.08201/) (2022)

## Abstract (arXiv)

Gaussian process (GP) models form a core part of probabilistic machine learning. Considerable research effort has been made into attacking three issues with GP models: how to compute efficiently when the number of data is large; how to approximate the posterior when the likelihood is not Gaussian and how to estimate covariance function parameter posteriors. This paper simultaneously addresses these, using a variational approximation to the posterior which is sparse in support of the function but otherwise free-form. The result is a Hybrid Monte-Carlo sampling scheme which allows for a non-Gaussian approximation over the function values and covariance parameters simultaneously, with efficient computations based on inducing-point sparse GPs. Code to replicate each experiment in this paper will be available shortly.