mlsanityData to 7 Oct 2026

Recommendations for Baselines and Benchmarking Approximate Gaussian Processes

arXiv 2402.09849 ↗First posted 15 Feb 2024cs.LG stat.MLcited by 2

Not on the map. Why?

Abstract (arXiv)

Gaussian processes (GPs) are a mature and widely-used component of the ML toolbox. One of their desirable qualities is automatic hyperparameter selection, which allows for training without user intervention. However, in many realistic settings, approximations are typically needed, which typically do require tuning. We argue that this requirement for tuning complicates evaluation, which has led to a lack of a clear recommendations on which method should be used in which situation. To address this, we make recommendations for comparing GP approximations based on a specification of what a user should expect from a method. In addition, we develop a training procedure for the variational method of Titsias [2009] that leaves no choices to the user, and show that this is a strong baseline that meets our specification. We conclude that benchmarking according to our suggestions gives a clearer view of the current state of the field, and uncovers problems that are still open that future papers should address.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 49 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

23 of 49 references.

References · 23

  1. 2016Variational Inference: A Review for Statisticianscited by 1,466
  2. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  3. 2013Gaussian Processes for Big Datacited by 377
  4. 2014Scalable Variational Gaussian Process Classificationcited by 226
  5. 2015Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)cited by 175
  6. 2016GPflow: A Gaussian process library using TensorFlowcited by 139
  7. 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 125
  8. 2019Exact Gaussian Processes on a Million Data Pointscited by 118
  9. 2021The Promises and Pitfalls of Deep Kernel Learningcited by 81
  10. 2016Understanding Probabilistic Sparse Gaussian Process Approximationscited by 68
  11. 2019Rates of Convergence for Sparse Variational Gaussian Process Regressioncited by 62
  12. 2015On Sparse variational methods and the Kullback-Leibler divergence between stochastic processescited by 56
11 more
  1. 2020Convergence of Sparse Variational Inference in Gaussian Processes Regressioncited by 53
  2. 2023Sampling from Gaussian Process Posteriors using Stochastic Gradient Descentcited by 27
  3. 2015Thoughts on Massively Scalable Gaussian Processescited by 26
  4. 2019Validated Variational Inference via Practical Posterior Error Boundscited by 26
  5. 2021Tighter Bounds on the Log Marginal Likelihood of Gaussian Process Regression Using Conjugate Gradientscited by 20
  6. 2014Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Modelscited by 19
  7. 2017Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processescited by 19
  8. 2021Bias-Free Scalable Gaussian Processes via Randomized Truncationscited by 15
  9. 2021When are Iterative Gaussian Processes Reliably Accurate?cited by 12
  10. 2023Large-Scale Gaussian Processes via Alternating Projectioncited by 11
  11. 2022Memory Safe Computations with XLA Compilercited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inferencecited by 8
  2. 2026Sequential sparse Gaussian process quantile regressioncited by 0