mlsanityData to 7 Oct 2026

Scalable Cross Validation Losses for Gaussian Process Models

arXiv 2105.11535 ↗First posted 24 May 2021stat.ML cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We introduce a simple and scalable method for training Gaussian process (GP) models that exploits cross-validation and nearest neighbor truncation. To accommodate binary and multi-class classification we leverage P`olya-Gamma auxiliary variables and variational inference. In an extensive empirical comparison with a number of alternative methods for scalable GP regression and classification, we find that our method offers fast training and excellent predictive performance. We argue that the good predictive performance can be traced to the non-parametric nature of the resulting predictive distributions as well as to the cross-validation loss, which provides robustness against model mis-specification.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 22 of its 42 references that are in the map. First posted on arXiv 24 May 2021.

Builds on

22 of 42 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Billion-scale similarity search with GPUscited by 2,477
  3. 2018Pyro: Deep Universal Probabilistic Programmingcited by 379
  4. 2013Gaussian Processes for Big Datacited by 377
  5. 2014Scalable Variational Gaussian Process Classificationcited by 226
  6. 2018When Gaussian Process Meets Big Data: A Review of Scalable GPscited by 205
  7. 2019Exact Gaussian Processes on a Million Data Pointscited by 118
  8. 2014Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasetscited by 71
  9. 2016Understanding Probabilistic Sparse Gaussian Process Approximationscited by 68
  10. 2017A general framework for Vecchia approximations of Gaussian processescited by 66
  11. 2013Local Gaussian process approximation for large computer experimentscited by 59
  12. 2019On the marginal likelihood and cross-validationcited by 36
10 more
  1. 2015Dependent Multinomial Models Made Easy: Stick Breaking with the P'olya-Gamma Augmentationcited by 25
  2. 2020Sparse within Sparse Gaussian Processes using Neighbor Informationcited by 18
  3. 2018Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentationcited by 17
  4. 2019Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentationcited by 16
  5. 2020Deep Sigma Point Processescited by 16
  6. 2020Direct loss minimization algorithms for sparse Gaussian processescited by 10
  7. 2014Bayesian leave-one-out cross-validation approximations for Gaussian latent variable modelscited by 5
  8. 2020Towards new cross-validation-based estimators for Gaussian process regression: efficient adjoint computation of gradientscited by 5
  9. 2016Differentially Private Gaussian Processescited by 2
  10. 2020Sparse Gaussian Processes via Parametric Families of Compactly-supported Kernelscited by 2

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 1; 2023: 1; 2024: 2.

6 papers.

  1. 2022Variational Nearest Neighbor Gaussian Processcited by 8
  2. 2024Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inferencecited by 8
  3. 2021Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processescited by 7
  4. 2021Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefitscited by 3
  5. 2024Interventional Processes for Causal Uncertainty Quantificationcited by 2
  6. 2023On the Calibration and Uncertainty with P'olya-Gamma Augmentation for Dialog Retrieval Modelscited by 0