# Posterior and Computational Uncertainty in Gaussian Processes

arXiv: https://arxiv.org/abs/2205.15449 · HTML: https://mlsanity.com/paper/2205.15449/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 74 references that are in the map.
- First posted on arXiv 30 May 2022.

## Builds on (most cited first, 30 of 34)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences](https://mlsanity.com/paper/1807.02582/) (2018)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Kernel Operations on the GPU, with Autodiff, without Memory Overflows](https://mlsanity.com/paper/2004.11127/) (2020)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [Understanding Probabilistic Sparse Gaussian Process Approximations](https://mlsanity.com/paper/1606.04820/) (2016)
- [A general framework for Vecchia approximations of Gaussian processes](https://mlsanity.com/paper/1708.06302/) (2017)
- [Pathwise Conditioning of Gaussian Processes](https://mlsanity.com/paper/2011.04026/) (2020)
- [Rates of Convergence for Sparse Variational Gaussian Process Regression](https://mlsanity.com/paper/1903.03571/) (2019)
- [Probabilistic Numerics and Uncertainty in Computations](https://mlsanity.com/paper/1506.01326/) (2015)
- [Bayesian Probabilistic Numerical Methods](https://mlsanity.com/paper/1702.03673/) (2017)
- [Constant-Time Predictive Distributions for Gaussian Processes](https://mlsanity.com/paper/1803.06058/) (2018)
- [Sparse Cholesky factorization by Kullback-Leibler minimization](https://mlsanity.com/paper/2004.14455/) (2020)
- [Preconditioning Kernel Matrices](https://mlsanity.com/paper/1602.06693/) (2016)
- [Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition](https://mlsanity.com/paper/1710.07324/) (2017)
- [Thoughts on Massively Scalable Gaussian Processes](https://mlsanity.com/paper/1511.01870/) (2015)
- [Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity](https://mlsanity.com/paper/1706.02205/) (2017)
- [Preconditioning for Scalable Gaussian Process Hyperparameter Optimization](https://mlsanity.com/paper/2107.00243/) (2021)
- [Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradients](https://mlsanity.com/paper/2102.08314/) (2021)
- [A la Carte - Learning Fast Kernels](https://mlsanity.com/paper/1412.6493/) (2014)
- [A Modern Retrospective on Probabilistic Numerics](https://mlsanity.com/paper/1901.04457/) (2019)
- [Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2106.01121/) (2021)
- [Bias-Free Scalable Gaussian Processes via Randomized Truncations](https://mlsanity.com/paper/2102.06695/) (2021)
- [Probabilistic Interpretation of Linear Solvers](https://mlsanity.com/paper/1402.2058/) (2014)
- [Probabilistic Linear Solvers for Machine Learning](https://mlsanity.com/paper/2010.09691/) (2020)
- [Scalable Gaussian Processes with Grid-Structured Eigenfunctions (GP-GRIEF)](https://mlsanity.com/paper/1807.02125/) (2018)
- [Scaling up Kernel Ridge Regression via Locality Sensitive Hashing](https://mlsanity.com/paper/2003.09756/) (2020)

## Built on by (most cited first, 24 of 24)

- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [Computation-Aware Kalman Filtering and Smoothing](https://mlsanity.com/paper/2405.08971/) (2024)
- [Large-Scale Gaussian Processes via Alternating Projection](https://mlsanity.com/paper/2310.17137/) (2023)
- [Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference](https://mlsanity.com/paper/2411.01036/) (2024)
- [Calibrated Computation-Aware Gaussian Processes](https://mlsanity.com/paper/2410.08796/) (2024)
- [Low-Budget Simulation-Based Inference with Bayesian Neural Networks](https://mlsanity.com/paper/2408.15136/) (2024)
- [Entity Aware Modelling: A Survey](https://mlsanity.com/paper/2302.08406/) (2023)
- [Accelerating Non-Conjugate Gaussian Processes By Trading Off Computation For Uncertainty](https://mlsanity.com/paper/2310.20285/) (2023)
- [Scaling up Probabilistic PDE Simulators with Structured Volumetric Information](https://mlsanity.com/paper/2406.05020/) (2024)
- [Adaptive sparse variational approximations for Gaussian process regression](https://mlsanity.com/paper/2504.03321/) (2025)
- [Wiener Chaos in Kernel Regression: Towards Untangling Aleatoric and Epistemic Uncertainty](https://mlsanity.com/paper/2312.07387/) (2023)
- [High-Dimensional Gaussian Process Regression with Soft Kernel Interpolation](https://mlsanity.com/paper/2410.21419/) (2024)
- [Online Conformal Probabilistic Numerics via Adaptive Edge-Cloud Offloading](https://mlsanity.com/paper/2503.14453/) (2025)
- [Randomised Postiterations for Calibrated BayesCG](https://mlsanity.com/paper/2504.04247/) (2025)
- [Robust and Computation-Aware Gaussian Processes](https://mlsanity.com/paper/2505.21133/) (2025)
- [PolicyGuard: Towards Test-time and Step-level Adversary (Backdoor) Defense for Reinforcement Learning Agent](https://mlsanity.com/paper/2606.12896/) (2026)
- [Kernel Multigrid: Accelerate Back-fitting via Sparse Gaussian Process Regression](https://mlsanity.com/paper/2403.13300/) (2024)
- [Gaussian Processes Sampling with Sparse Grids under Additive Schwarz Preconditioner](https://mlsanity.com/paper/2408.00206/) (2024)
- [Learning to Solve Related Linear Systems](https://mlsanity.com/paper/2503.17265/) (2025)
- [Simplex-to-Euclidean Bijection for Conjugate and Calibrated Multiclass Gaussian Process Classification](https://mlsanity.com/paper/2603.16621/) (2026)
- [Affine Tracing: A New Paradigm for Probabilistic Linear Solvers](https://mlsanity.com/paper/2605.10566/) (2026)
- [Computation-Aware Kalman Filtering with Model Selection for Neural Dynamics](https://mlsanity.com/paper/2606.01468/) (2026)
- [Adaptive multi-resolution Gaussian processes: Scalable exact inference with naturally data-sparse covariance matrices](https://mlsanity.com/paper/2609.30348/) (2026)
- [Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models](https://mlsanity.com/paper/2610.03647/) (2026)

## Abstract (arXiv)

Gaussian processes scale prohibitively with the size of the dataset. In response, many approximation methods have been developed, which inevitably introduce approximation error. This additional source of uncertainty, due to limited computation, is entirely ignored when using the approximate posterior. Therefore in practice, GP models are often as much about the approximation method as they are about the data. Here, we develop a new class of methods that provides consistent estimation of the combined uncertainty arising from both the finite number of data observed and the finite amount of computation expended. The most common GP approximations map to an instance in this class, such as methods based on the Cholesky factorization, conjugate gradients, and inducing points. For any method in this class, we prove (i) convergence of its posterior mean in the associated RKHS, (ii) decomposability of its combined posterior covariance into mathematical and computational covariances, and (iii) that the combined variance is a tight worst-case bound for the squared error between the method's posterior mean and the latent function. Finally, we empirically demonstrate the consequences of ignoring computational uncertainty and show how implicitly modeling it improves generalization performance on benchmark datasets.