# New Bounds for Sparse Variational Gaussian Processes

arXiv: https://arxiv.org/abs/2502.08730 · HTML: https://mlsanity.com/paper/2502.08730/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 26 of 26)

- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Variational Fourier features for Gaussian processes](https://mlsanity.com/paper/1611.06740/) (2016)
- [On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes](https://mlsanity.com/paper/1504.07027/) (2015)
- [Convergence of Sparse Variational Inference in Gaussian Processes Regression](https://mlsanity.com/paper/2008.00323/) (2020)
- [A Case Study Competition Among Methods for Analyzing Large Spatial Data](https://mlsanity.com/paper/1710.05013/) (2017)
- [Product Kernel Interpolation for Scalable Gaussian Processes](https://mlsanity.com/paper/1802.08903/) (2018)
- [A Tutorial on Sparse Gaussian Processes and Variational Inference](https://mlsanity.com/paper/2012.13962/) (2020)
- [Variational Inference for Gaussian Process Modulated Poisson Processes](https://mlsanity.com/paper/1411.0254/) (2014)
- [Variational Inference for Gaussian Process Models with Linear Complexity](https://mlsanity.com/paper/1711.10127/) (2017)
- [Multi-task Learning for Aggregated Data using Gaussian Processes](https://mlsanity.com/paper/1906.09412/) (2019)
- [Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradients](https://mlsanity.com/paper/2102.08314/) (2021)
- [Orthogonally Decoupled Variational Gaussian Processes](https://mlsanity.com/paper/1809.08820/) (2018)
- [Sparse within Sparse Gaussian Processes using Neighbor Information](https://mlsanity.com/paper/2011.05041/) (2020)
- [Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2106.01121/) (2021)
- [Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)](https://mlsanity.com/paper/1807.02125/) (2018)
- [Efficient Optimization for Sparse Gaussian Process Regression](https://mlsanity.com/paper/1310.6007/) (2013)
- [Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition](https://mlsanity.com/paper/2106.05992/) (2021)
- [Deep Gaussian Processes with Decoupled Inducing Inputs](https://mlsanity.com/paper/1801.02939/) (2018)
- [Tighter sparse variational Gaussian processes](https://mlsanity.com/paper/2502.04750/) (2025)
- [Spherical Inducing Features for Orthogonally-Decoupled Gaussian Processes](https://mlsanity.com/paper/2304.14034/) (2023)

## Built on by (most cited first, 2 of 2)

- [Sparse Gaussian Neural Processes](https://mlsanity.com/paper/2504.01650/) (2025)
- [Transformed Latent Variable Multi-Output Gaussian Processes](https://mlsanity.com/paper/2605.05133/) (2026)

## Abstract (arXiv)

Sparse variational Gaussian processes (GPs) construct tractable posterior approximations to GP models. At the core of these methods is the assumption that the true posterior distribution over training function values bf f and inducing variables bf u is approximated by a variational distribution that incorporates the conditional GP prior p(bf f | bf u) in its factorization. While this assumption is considered as fundamental, we show that for model training we can relax it through the use of a more general variational distribution q(bf f | bf u) that depends on N extra parameters, where N is the number of training examples. In GP regression, we can analytically optimize the evidence lower bound over the extra parameters and express a tractable collapsed bound that is tighter than the previous bound. The new bound is also amenable to stochastic optimization and its implementation requires minor modifications to existing sparse GP code. Further, we also describe extensions to non-Gaussian likelihoods. On several datasets we demonstrate that our method can reduce bias when learning the hyperparameters and can lead to better predictive performance.