mlsanityData to 7 Oct 2026

Variational Nearest Neighbor Gaussian Process

arXiv 2202.01694 ↗First posted 3 Feb 2022cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

Variational approximations to Gaussian processes (GPs) typically use a small set of inducing points to form a low-rank approximation to the covariance matrix. In this work, we instead exploit a sparse approximation of the precision matrix. We propose variational nearest neighbor Gaussian process (VNNGP), which introduces a prior that only retains correlations within K nearest-neighboring observations, thereby inducing sparse precision structure. Using the variational framework, VNNGP's objective can be factorized over both observations and inducing points, enabling stochastic optimization with a time complexity of O(K3). Hence, we can arbitrarily scale the inducing point size, even to the point of putting inducing points at every observed location. We compare VNNGP to other scalable GPs through various experiments, and demonstrate that VNNGP (1) can dramatically outperform low-rank methods, and (2) is less prone to overfitting than other nearest neighbor methods.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 3 Feb 2022.

Builds on

25 of 48 references.

References · 25

  1. 2017Billion-scale similarity search with GPUscited by 2,477
  2. 2013Gaussian Processes for Big Datacited by 377
  3. 2013Black Box Variational Inferencecited by 317
  4. 2015Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)cited by 175
  5. 2019Exact Gaussian Processes on a Million Data Pointscited by 118
  6. 2016Variational Fourier features for Gaussian processescited by 85
  7. 2014Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasetscited by 71
  8. 2016Understanding Probabilistic Sparse Gaussian Process Approximationscited by 68
  9. 2017A general framework for Vecchia approximations of Gaussian processescited by 66
  10. 2019Rates of Convergence for Sparse Variational Gaussian Process Regressioncited by 62
  11. 2013Local Gaussian process approximation for large computer experimentscited by 59
  12. 2020A Framework for Interdomain and Multioutput Gaussian Processescited by 47
13 more
  1. 2015MCMC for Variationally Sparse Gaussian Processescited by 44
  2. 2018Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Modelscited by 38
  3. 2020Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimizationcited by 26
  4. 2016Permutation and Grouping Methods for Sharpening Gaussian Process Approximationscited by 24
  5. 2017Variational Inference for Gaussian Process Models with Linear Complexitycited by 21
  6. 2020Sparse within Sparse Gaussian Processes using Neighbor Informationcited by 18
  7. 2021Bias-Free Scalable Gaussian Processes via Randomized Truncationscited by 15
  8. 2021Correlation-based sparse inverse Cholesky factorization for fast Gaussian-process inferencecited by 12
  9. 2017Patchwork Kriging for Large-scale Gaussian Process Regressioncited by 9
  10. 2018Large-Scale Cox Process Inference using Variational Fourier Featurescited by 8
  11. 2021Hierarchical Inducing Point Gaussian Process for Inter-domain Observationscited by 6
  12. 2021Scalable Cross Validation Losses for Gaussian Process Modelscited by 6
  13. 2021Input Dependent Sparse Gaussian Processescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 0; 2025: 1; 2026: 1.

8 papers.

  1. 2022Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processescited by 11
  2. 2023Variational sparse inverse Cholesky approximation for latent Gaussian processes via double Kullback-Leibler minimizationcited by 4
  3. 2023Actually Sparse Variational Gaussian Processescited by 4
  4. 2022B'ezier Gaussian Processes for Tall and Wide Datacited by 2
  5. 2023Beyond Intuition, a Framework for Applying GPs to Real-World Datacited by 1
  6. 2023Trigonometric Quadrature Fourier Features for Scalable Gaussian Process Regressioncited by 1
  7. 2025A Hierarchical Variational Graph Fused Lasso for Recovering Relative Rates in Spatial Compositional Datacited by 0
  8. 2026Adaptive multi-resolution Gaussian processes: Scalable exact inference with naturally data-sparse covariance matricescited by 0