# Leveraging Locality and Robustness to Achieve Massively Scalable Gaussian Process Regression

arXiv: https://arxiv.org/abs/2306.14731 · HTML: https://mlsanity.com/paper/2306.14731/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 26 Jun 2023.

## Builds on (most cited first, 16 of 16)

- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)](https://mlsanity.com/paper/1503.01057/) (2015)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Distribution Calibration for Regression](https://mlsanity.com/paper/1905.06023/) (2019)
- [Distributed Gaussian Processes](https://mlsanity.com/paper/1502.02843/) (2015)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictions](https://mlsanity.com/paper/1410.7827/) (2014)
- [Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1806.00720/) (2018)
- [Product Kernel Interpolation for Scalable Gaussian Processes](https://mlsanity.com/paper/1802.08903/) (2018)
- [Approximate Inference for Fully Bayesian Gaussian Process Regression](https://mlsanity.com/paper/1912.13440/) (2019)
- [Sparse within Sparse Gaussian Processes using Neighbor Information](https://mlsanity.com/paper/2011.05041/) (2020)
- [Faster Kernel Interpolation for Gaussian Processes](https://mlsanity.com/paper/2101.11751/) (2021)
- [On Random Subsampling of Gaussian Process Regression: A Graphon-Based Analysis](https://mlsanity.com/paper/1901.09541/) (2019)
- [Provably Reliable Large-Scale Sampling from Gaussian Processes](https://mlsanity.com/paper/2211.08036/) (2022)

## Built on by (most cited first, 5 of 5)

- [Neighbour-Driven Gaussian Process Variational Autoencoders for Scalable Structured Latent Modelling](https://mlsanity.com/paper/2505.16481/) (2025)
- [Further Understanding of a Local Gaussian Process Approximation: Characterising Convergence in the Finite Regime](https://mlsanity.com/paper/2404.06200/) (2024)
- [Efficient Two-Stage Gaussian Process Regression Via Automatic Kernel Search and Subsampling](https://mlsanity.com/paper/2405.13785/) (2024)
- [Preconditioned Additive Gaussian Processes with Fourier Acceleration](https://mlsanity.com/paper/2504.00480/) (2025)
- [Scalable Gaussian process inference via neural feature maps](https://mlsanity.com/paper/2605.10285/) (2026)

## Abstract (arXiv)

The accurate predictions and principled uncertainty measures provided by GP regression incur O(n^3) cost which is prohibitive for modern-day large-scale applications. This has motivated extensive work on computationally efficient approximations. We introduce a new perspective by exploring robustness properties and limiting behaviour of GP nearest-neighbour (GPnn) prediction. We demonstrate through theory and simulation that as the data-size n increases, accuracy of estimated parameters and GP model assumptions become increasingly irrelevant to GPnn predictive accuracy. Consequently, it is sufficient to spend small amounts of work on parameter estimation in order to achieve high MSE accuracy, even in the presence of gross misspecification. In contrast, as n tends to infinity, uncertainty calibration and NLL are shown to remain sensitive to just one parameter, the additive noise-variance; but we show that this source of inaccuracy can be corrected for, thereby achieving both well-calibrated uncertainty measures and accurate predictions at remarkably low computational cost. We exhibit a very simple GPnn regression algorithm with stand-out performance compared to other state-of-the-art GP approximations as measured on large UCI datasets. It operates at a small fraction of those other methods' training costs, for example on a basic laptop taking about 30 seconds to train on a dataset of size n = 1.6 x 10^6.