mlsanityData to 7 Oct 2026

On the Laplace Approximation as Model Selection Criterion for Gaussian Processes

arXiv 2403.09215 ↗First posted 14 Mar 2024cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Model selection aims to find the best model in terms of accuracy, interpretability or simplicity, preferably all at once. In this work, we focus on evaluating model performance of Gaussian process models, i.e. finding a metric that provides the best trade-off between all those criteria. While previous work considers metrics like the likelihood, AIC or dynamic nested sampling, they either lack performance or have significant runtime issues, which severely limits applicability. We address these challenges by introducing multiple metrics based on the Laplace approximation, where we overcome a severe inconsistency occuring during naive application of the Laplace approximation. Experiments show that our metrics are comparable in quality to the gold standard dynamic nested sampling without compromising for computational speed. Our model selection criteria allow significantly faster and high quality model selection of Gaussian process models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 45 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

18 of 45 references.

References · 18

  1. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  2. 2018Online Structured Laplace Approximations For Overcoming Catastrophic Forgettingcited by 197
  3. 2013Gaussian Process Kernels for Pattern Discovery and Extrapolationcited by 131
  4. 2013Structure Discovery in Nonparametric Regression through Compositional Kernel Searchcited by 86
  5. 2020Mat'ern Gaussian processes on Riemannian manifoldscited by 82
  6. 2017Linearly constrained Gaussian processescited by 41
  7. 2014Automatic Construction and Natural-Language Description of Nonparametric Regression Modelscited by 40
  8. 2020Learnable Uncertainty under Laplace Approximationscited by 29
  9. 2020Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processescited by 29
  10. 2017Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processescited by 19
  11. 2022Gaussian Process Priors for Systems of Linear Partial Differential Equations with Constant Coefficientscited by 17
  12. 2022Maximum Likelihood Estimation in Gaussian Process Regression is Ill-Posedcited by 16
6 more
  1. 2017Dynamic nested sampling: an improved algorithm for parameter estimation and evidence calculationcited by 15
  2. 2019Implicit Kernel Learningcited by 13
  3. 2020Marginalised Gaussian Processes with Nested Samplingcited by 7
  4. 2022NCVX: A General-Purpose Optimization Solver for Constrained Machine and Deep Learningcited by 7
  5. 2019Vecchia-Laplace approximations of generalized Gaussian processes for big non-Gaussian spatial datacited by 4
  6. 2023Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Modelscited by 3

Built on by

Not cited yet.