# On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach

arXiv: https://arxiv.org/abs/2103.03321 · HTML: https://mlsanity.com/paper/2103.03321/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 71 references that are in the map.
- First posted on arXiv 4 Mar 2021.

## Builds on (most cited first, 22 of 22)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Rates of Convergence for Sparse Variational Gaussian Process Regression](https://mlsanity.com/paper/1903.03571/) (2019)
- [On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes](https://mlsanity.com/paper/1504.07027/) (2015)
- [On Markov chain Monte Carlo methods for tall data](https://mlsanity.com/paper/1505.02827/) (2015)
- [Deep Gaussian Processes for Multi-fidelity Modeling](https://mlsanity.com/paper/1903.07320/) (2019)
- [MCMC for Variationally Sparse Gaussian Processes](https://mlsanity.com/paper/1506.04000/) (2015)
- [A Case Study Competition Among Methods for Analyzing Large Spatial Data](https://mlsanity.com/paper/1710.05013/) (2017)
- [Firefly Monte Carlo: Exact MCMC with Subsets of Data](https://mlsanity.com/paper/1403.5693/) (2014)
- [Speeding Up MCMC by Efficient Data Subsampling](https://mlsanity.com/paper/1404.4178/) (2014)
- [On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods](https://mlsanity.com/paper/1306.4032/) (2013)
- [Banded Matrix Operators for Gaussian Markov Models in the Automatic Differentiation Era](https://mlsanity.com/paper/1902.10078/) (2019)
- [Parallelizable sparse inverse formulation Gaussian processes (SpInGP)](https://mlsanity.com/paper/1610.08035/) (2016)
- [Hamiltonian Monte Carlo with Energy Conserving Subsampling](https://mlsanity.com/paper/1708.00955/) (2017)
- [Probabilistic selection of inducing points in sparse Gaussian processes](https://mlsanity.com/paper/2010.09370/) (2020)
- [Gaussian Process Volatility Model](https://mlsanity.com/paper/1402.3085/) (2014)
- [Pseudo-Marginal Bayesian Inference for Gaussian Processes](https://mlsanity.com/paper/1310.0740/) (2013)
- [Computer emulation with non-stationary Gaussian processes](https://mlsanity.com/paper/1308.4756/) (2013)
- [Diagnostic-Driven Nonstationary Emulators Using Kernel Mixtures](https://mlsanity.com/paper/1803.04906/) (2018)
- [The block-Poisson estimator for optimally tuned exact subsampling MCMC](https://mlsanity.com/paper/1603.08232/) (2016)
- [Bayesian Inference for Gaussian Process Classifiers with Annealing and Pseudo-Marginal MCMC](https://mlsanity.com/paper/1311.7320/) (2013)
- [Pseudo-marginal Bayesian inference for supervised Gaussian process latent variable models](https://mlsanity.com/paper/1803.10746/) (2018)

## Built on by (most cited first, 1 of 1)

- [A Bayesian Approach To Graph Partitioning](https://mlsanity.com/paper/2204.12927/) (2022)

## Abstract (arXiv)

Gaussian processes (GPs) are frequently used in machine learning and statistics to construct powerful models. However, when employing GPs in practice, important considerations must be made, regarding the high computational burden, approximation of the posterior, choice of the covariance function and inference of its hyperparmeters. To address these issues, Hensman et al. (2015) combine variationally sparse GPs with Markov chain Monte Carlo (MCMC) to derive a scalable, flexible and general framework for GP models. Nevertheless, the resulting approach requires intractable likelihood evaluations for many observation models. To bypass this problem, we propose a pseudo-marginal (PM) scheme that offers asymptotically exact inference as well as computational gains through doubly stochastic estimators for the intractable likelihood and large datasets. In complex models, the advantages of the PM scheme are particularly evident, and we demonstrate this on a two-level GP regression model with a nonparametric covariance function to capture non-stationarity.