# Doubly Sparse Variational Gaussian Processes

arXiv: https://arxiv.org/abs/2001.05363 · HTML: https://mlsanity.com/paper/2001.05363/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2020.

## Built on by (most cited first, 14 of 14)

- [Spatio-Temporal Variational Gaussian Processes](https://mlsanity.com/paper/2111.01732/) (2021)
- [Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2106.01121/) (2021)
- [Markovian Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2207.05543/) (2022)
- [Sparse Algorithms for Markovian Gaussian Processes](https://mlsanity.com/paper/2103.10710/) (2021)
- [Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processes](https://mlsanity.com/paper/2106.10210/) (2021)
- [Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trains](https://mlsanity.com/paper/2306.01802/) (2023)
- [The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions](https://mlsanity.com/paper/2306.07774/) (2023)
- [Gaussian Processes on Hypergraphs](https://mlsanity.com/paper/2106.01982/) (2021)
- [Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling](https://mlsanity.com/paper/2505.16481/) (2025)
- [Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference](https://mlsanity.com/paper/2407.02476/) (2024)
- [Spatiotemporal modeling of European paleoclimate using doubly sparse Gaussian processes](https://mlsanity.com/paper/2211.08160/) (2022)
- [Deep Gaussian Markov Random Fields for Graph-Structured Dynamical Systems](https://mlsanity.com/paper/2306.08445/) (2023)
- [Review of Recent Advances in Gaussian Process Regression Methods](https://mlsanity.com/paper/2409.08112/) (2024)
- [Designing an Optimal Sensor Network via Minimizing Information Loss](https://mlsanity.com/paper/2512.05940/) (2025)

## Abstract (arXiv)

The use of Gaussian process models is typically limited to datasets with a few tens of thousands of observations due to their complexity and memory footprint. The two most commonly used methods to overcome this limitation are 1) the variational sparse approximation which relies on inducing points and 2) the state-space equivalent formulation of Gaussian processes which can be seen as exploiting some sparsity in the precision matrix. We propose to take the best of both worlds: we show that the inducing point framework is still valid for state space models and that it can bring further computational and memory savings. Furthermore, we provide the natural gradient formulation for the proposed variational parameterisation. Finally, this work makes it possible to use the state-space formulation inside deep Gaussian process models as illustrated in one of the experiments.