mlsanityData to 7 Oct 2026

Convergence of Sparse Variational Inference in Gaussian Processes Regression

arXiv 2008.00323 ↗First posted 1 Aug 2020stat.ML cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Gaussian processes are distributions over functions that are versatile and mathematically convenient priors in Bayesian modelling. However, their use is often impeded for data with large numbers of observations, N, due to the cubic (in N) cost of matrix operations used in exact inference. Many solutions have been proposed that rely on M ll N inducing variables to form an approximation at a cost of O(NM2). While the computational cost appears linear in N, the true complexity depends on how M must scale with N to ensure a certain quality of the approximation. In this work, we investigate upper and lower bounds on how M needs to grow with N to ensure high quality approximations. We show that we can make the KL-divergence between the approximate model and the exact posterior arbitrarily small for a Gaussian-noise regression model with Mll N. Specifically, for the popular squared exponential kernel and D-dimensional Gaussian distributed covariates, M=O((log N)D) suffice and a method with an overall computational cost of O(N(log N)2D(loglog N)2) can be used to perform inference.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 6; 2023: 8; 2024: 9; 2025: 10; 2026: 11.

53 papers.

Sort:

By area:

  1. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  2. 2024CiteME: Can Language Models Accurately Cite Scientific Claims?cited by 45
  3. 2023Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisationcited by 32
  4. 2021SigGPDE: Scaling Sparse Gaussian Processes on Sequential Datacited by 22
  5. 2021Last Layer Marginal Likelihood for Invariance Learningcited by 21
  6. 2021Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradientscited by 20
  7. 2022Improved Convergence Rates for Sparse Approximation Methods in Kernel-Based Learningcited by 18
  8. 2021Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processescited by 17
  9. 2021Robust Uncertainty Bounds in Reproducing Kernel Hilbert Spaces: A Convex Optimization Approachcited by 14
  10. 2022Efficient Hyperparameter Tuning for Large Scale Kernel Ridge Regressioncited by 11
  11. 2023Memory-Based Dual Gaussian Processes for Sequential Learningcited by 11
  12. 2024COMO: Compact Mapping and Odometrycited by 10
38 more
  1. 2024Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inferencecited by 8
  2. 2024Scalable Bayesian Optimization via Focalized Sparse Gaussian Processescited by 8
  3. 2022Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Treescited by 5
  4. 2024Continuous Bayesian Model Selection for Multivariate Causal Discoverycited by 5
  5. 2021Gaussian Processes on Hypergraphscited by 4
  6. 2022Sparse Gaussian Process Hyperparameters: Optimize or Integrate?cited by 4
  7. 2022Memory Safe Computations with XLA Compilercited by 3
  8. 2023Fast post-process Bayesian inference with Variational Sparse Bayesian Quadraturecited by 3
  9. 2023Integrated Variational Fourier Features for Fast Spatial Modelling with Gaussian Processescited by 3
  10. 2022B'ezier Gaussian Processes for Tall and Wide Datacited by 2
  11. 2023Evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systemscited by 2
  12. 2023Gradient and Uncertainty Enhanced Sequential Sampling for Global Fitcited by 2
  13. 2024Recommendations for Baselines and Benchmarking Approximate Gaussian Processescited by 2
  14. 2025New Bounds for Sparse Variational Gaussian Processescited by 2
  15. 2025Efficient Transformed Gaussian Process State-Space Models for Non-Stationary High-Dimensional Dynamical Systemscited by 2
  16. 2025Adaptive sparse variational approximations for Gaussian process regressioncited by 2
  17. 2025Bayesian Data Sketching for Varying Coefficient Regression Modelscited by 2
  18. 2026Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Modelscited by 2
  19. 2026Incremental Transformer Neural Processescited by 2
  20. 2023Efficient Sensor Placement from Regression with Sparse Gaussian Processes in Continuous and Discrete Spacescited by 1
  21. 2023Pointwise uncertainty quantification for sparse variational Gaussian process regression with a Brownian motion priorcited by 1
  22. 2024Adjusting Model Size in Continual Gaussian Processes: How Big is Big Enough?cited by 1
  23. 2026Knowledge Gradient for Preference Learningcited by 1
  24. 2026Three Costs of Amortizing Gaussian Process Inference with Neural Processescited by 1
  25. 2021Barely Biased Learning for Gaussian Process Regressioncited by 0
  26. 2021How Good are Low-Rank Approximations in Gaussian Process Regression?cited by 0
  27. 2024Further Understanding of a Local Gaussian Process Approximation: Characterising Convergence in the Finite Regimecited by 0
  28. 2024Reservoir History Matching of the Norne field with generative exotic priors and a coupled Mixture of Experts -- Physics Informed Neural Operator Forward Modelcited by 0
  29. 2025Low-rank computation of the posterior mean in Multi-Output Gaussian Processescited by 0
  30. 2025Sparse Techniques for Regression in Deep Gaussian Processescited by 0
  31. 2025Taking the GP Out of the Loopcited by 0
  32. 2025Sparse Gaussian Processes: Structured Approximations and Power-EP Revisitedcited by 0
  33. 2025Decentralized Gaussian Process Classification and an Application in Subsea Roboticscited by 0
  34. 2025No-Regret Gaussian Process Optimization of Time-Varying Functionscited by 0
  35. 2026Joint Parameter and State-Space Bayesian Optimization: Using Process Expertise to Accelerate Manufacturing Optimizationcited by 0
  36. 2026A Distributed Gaussian Process Model for Multi-Robot Mappingcited by 0
  37. 2026Inverse-Free Sparse Variational Gaussian Processescited by 0
  38. 2026Scalable Gaussian process inference via neural feature mapscited by 0

50 most cited of 53 All 53 (JSON)