# Understanding Probabilistic Sparse Gaussian Process Approximations

arXiv: https://arxiv.org/abs/1606.04820 · HTML: https://mlsanity.com/paper/1606.04820/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2016.

## Built on by (most cited first, 30 of 68)

- [How Powerful are Performance Predictors in Neural Architecture Search?](https://mlsanity.com/paper/2104.01177/) (2021)
- [Spatio-Temporal Variational Gaussian Processes](https://mlsanity.com/paper/2111.01732/) (2021)
- [Posterior and Computational Uncertainty in Gaussian Processes](https://mlsanity.com/paper/2205.15449/) (2022)
- [Meta-Learning Reliable Priors in the Function Space](https://mlsanity.com/paper/2106.03195/) (2021)
- [Conditioning Sparse Variational Gaussian Processes for Online Decision-making](https://mlsanity.com/paper/2110.15172/) (2021)
- [Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradients](https://mlsanity.com/paper/2102.08314/) (2021)
- [Kernel Interpolation for Scalable Online Gaussian Processes](https://mlsanity.com/paper/2103.01454/) (2021)
- [Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2106.01121/) (2021)
- [Bias-Free Scalable Gaussian Processes via Randomized Truncations](https://mlsanity.com/paper/2102.06695/) (2021)
- [Learning a Depth Covariance Function](https://mlsanity.com/paper/2303.12157/) (2023)
- [Grokking Beyond Neural Networks: An Empirical Exploration with Model Complexity](https://mlsanity.com/paper/2310.17247/) (2023)
- [Surprisingly Strong Performance Prediction with Neural Graph Features](https://mlsanity.com/paper/2404.16551/) (2024)
- [Bayesian Calibration of Imperfect Computer Models using Physics-Informed Priors](https://mlsanity.com/paper/2201.06463/) (2022)
- [Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference](https://mlsanity.com/paper/2411.01036/) (2024)
- [Robust and Conjugate Spatio-Temporal Gaussian Processes](https://mlsanity.com/paper/2502.02450/) (2025)
- [Adaptive Inducing Points Selection For Gaussian Processes](https://mlsanity.com/paper/2107.10066/) (2021)
- [Variational Nearest Neighbor Gaussian Process](https://mlsanity.com/paper/2202.01694/) (2022)
- [Free-Form Variational Inference for Gaussian Process State-Space Models](https://mlsanity.com/paper/2302.09921/) (2023)
- [Hierarchical Inducing Point Gaussian Process for Inter-domain Observations](https://mlsanity.com/paper/2103.00393/) (2021)
- [Scalable Cross Validation Losses for Gaussian Process Models](https://mlsanity.com/paper/2105.11535/) (2021)
- [Uncertainty quantification of a three-dimensional in-stent restenosis model with surrogate modelling](https://mlsanity.com/paper/2111.06173/) (2021)
- [Bridging the reality gap in quantum devices with physics-aware machine learning](https://mlsanity.com/paper/2111.11285/) (2021)
- [Continuous Bayesian Model Selection for Multivariate Causal Discovery](https://mlsanity.com/paper/2411.10154/) (2024)
- [Adversarial Robustness Guarantees for Gaussian Processes](https://mlsanity.com/paper/2104.03180/) (2021)
- [Input Dependent Sparse Gaussian Processes](https://mlsanity.com/paper/2107.07281/) (2021)
- [Revisiting Active Sets for Gaussian Process Decoders](https://mlsanity.com/paper/2209.04636/) (2022)
- [Learning Choice Functions with Gaussian Processes](https://mlsanity.com/paper/2302.00406/) (2023)
- [Scalable Gaussian Processes with Latent Kronecker Structure](https://mlsanity.com/paper/2506.06895/) (2025)
- [Deep Gaussian Processes for Biogeophysical Parameter Retrieval and Model Inversion](https://mlsanity.com/paper/2104.10638/) (2021)
- [Memory Safe Computations with XLA Compiler](https://mlsanity.com/paper/2206.14148/) (2022)

## Abstract (arXiv)

Good sparse approximations are essential for practical inference in Gaussian Processes as the computational cost of exact methods is prohibitive for large datasets. The Fully Independent Training Conditional (FITC) and the Variational Free Energy (VFE) approximations are two recent popular methods. Despite superficial similarities, these approximations have surprisingly different theoretical properties and behave differently in practice. We thoroughly investigate the two methods for regression both analytically and through illustrative examples, and draw conclusions to guide practical application.