mlsanityData to 7 Oct 2026

Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference

arXiv 2411.01036 ↗First posted 1 Nov 2024cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

Model selection in Gaussian processes scales prohibitively with the size of the training dataset, both in time and memory. While many approximations exist, all incur inevitable approximation error. Recent work accounts for this error in the form of computational uncertainty, which enables -- at the cost of quadratic complexity -- an explicit tradeoff between computation and precision. Here we extend this development to model selection, which requires significant enhancements to the existing approach, including linear-time scaling in the size of the dataset. We propose a novel training loss for hyperparameter optimization and demonstrate empirically that the resulting method can outperform SGPR, CGGP and SVGP, state-of-the-art methods for GP model selection, on medium to large-scale datasets. Our experiments show that model selection for computation-aware GPs trained on 1.8 million data points can be done within a few hours on a single GPU. As a result of this work, Gaussian processes can be trained on large-scale datasets without significantly compromising their ability to quantify uncertainty -- a fundamental prerequisite for optimal decision-making.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 37 of its 62 references that are in the map. First posted on arXiv 1 Nov 2024.

Builds on

37 of 62 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  4. 2013Gaussian Processes for Big Datacited by 377
  5. 2014Scalable Variational Gaussian Process Classificationcited by 226
  6. 2018When Gaussian Process Meets Big Data: A Review of Scalable GPscited by 205
  7. 2018Gaussian Processes and Kernel Methods: A Review on Connections and Equivalencescited by 188
  8. 2015On the Equivalence between Kernel Quadrature Rules and Random Feature Expansionscited by 136
  9. 2019Exact Gaussian Processes on a Million Data Pointscited by 118
  10. 2017Batched Large-scale Bayesian Optimization in High-dimensional Spacescited by 116
  11. 2016Variational Fourier features for Gaussian processescited by 85
  12. 2014Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasetscited by 71
25 more
  1. 2016Understanding Probabilistic Sparse Gaussian Process Approximationscited by 68
  2. 2017A general framework for Vecchia approximations of Gaussian processescited by 66
  3. 2019Rates of Convergence for Sparse Variational Gaussian Process Regressioncited by 62
  4. 2015Probabilistic Numerics and Uncertainty in Computationscited by 59
  5. 2020Convergence of Sparse Variational Inference in Gaussian Processes Regressioncited by 53
  6. 2017Bayesian Probabilistic Numerical Methodscited by 50
  7. 2018Constant-Time Predictive Distributions for Gaussian Processescited by 48
  8. 2020A Framework for Interdomain and Multioutput Gaussian Processescited by 47
  9. 2020Sparse Gaussian Processes with Spherical Harmonic Featurescited by 43
  10. 2022Physics-Informed Gaussian Process Regression Generalizes Linear PDE Solverscited by 40
  11. 2020Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimizationcited by 26
  12. 2022Posterior and Computational Uncertainty in Gaussian Processescited by 23
  13. 2021Preconditioning for Scalable Gaussian Process Hyperparameter Optimizationcited by 22
  14. 2021Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradientscited by 20
  15. 2018Orthogonally Decoupled Variational Gaussian Processescited by 19
  16. 2021Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processescited by 17
  17. 2021Bias-Free Scalable Gaussian Processes via Randomized Truncationscited by 15
  18. 2014Probabilistic Interpretation of Linear Solverscited by 12
  19. 2024Computation-Aware Kalman Filtering and Smoothingcited by 12
  20. 2020Direct loss minimization algorithms for sparse Gaussian processescited by 10
  21. 2020Probabilistic Linear Solvers for Machine Learningcited by 10
  22. 2018A Bayesian Conjugate Gradient Methodcited by 7
  23. 2021Scalable Cross Validation Losses for Gaussian Process Modelscited by 6
  24. 2024Calibrated Computation-Aware Gaussian Processescited by 6
  25. 2024Recommendations for Baselines and Benchmarking Approximate Gaussian Processescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.

  1. 2025Scalable Gaussian Processes with Latent Kronecker Structurecited by 4
  2. 2026Conditioning Gaussian Processes on Almost Anythingcited by 3
  3. 2025Randomised Postiterations for Calibrated BayesCGcited by 1
  4. 2025Robust and Computation-Aware Gaussian Processescited by 1
  5. 2025Ksurf-Drone: Attention Kalman Filter for Contextual Bandit Optimization in Cloud Resource Allocationcited by 1
  6. 2025Learning to Solve Related Linear Systemscited by 0
  7. 2026Affine Tracing: A New Paradigm for Probabilistic Linear Solverscited by 0
  8. 2026Computation-Aware Kalman Filtering with Model Selection for Neural Dynamicscited by 0