mlsanityData to 7 Oct 2026

Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences

arXiv 1807.02582 ↗First posted 6 Jul 2018stat.ML cs.LGcited by 188

Not on the map. Why?

Abstract (arXiv)

This paper is an attempt to bridge the conceptual gaps between researchers working on the two widely used approaches based on positive definite kernels: Bayesian learning or inference using Gaussian processes on the one side, and frequentist kernel methods based on reproducing kernel Hilbert spaces on the other. It is widely known in machine learning that these two formalisms are closely related; for instance, the estimator of kernel ridge regression is identical to the posterior mean of Gaussian process regression. However, they have been studied and developed almost independently by two essentially separate communities, and this makes it difficult to seamlessly transfer results between them. Our aim is to overcome this potential difficulty. To this end, we review several old and new results and concepts from either side, and juxtapose algorithmic quantities from each framework to highlight close similarities. We also provide discussions on subtle philosophical and theoretical differences between the two approaches.

Key facts. Cited by 188 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 37; 2022: 34; 2023: 31; 2024: 29; 2025: 34; 2026: 23.

188 papers.

Sort:

By area:

  1. 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 95
  2. 2021Optimal Order Simple Regret for Gaussian Process Banditscited by 52
  3. 2021Misspecified Gaussian Process Bandit Optimizationcited by 52
  4. 2022Optimal Rates for Regularized Conditional Mean Embedding Learningcited by 50
  5. 2021Practical and Rigorous Uncertainty Bounds for Gaussian Process Regressioncited by 49
  6. 2022Physics-Informed Gaussian Process Regression Generalizes Linear PDE Solverscited by 40
  7. 2021Weighted Gaussian Process Bandits for Non-stationary Environmentscited by 30
  8. 2021Deep Neural Networks as Point Estimates for Deep Gaussian Processescited by 26
  9. 2021Locality defeats the curse of dimensionality in convolutional teacher-student scenarioscited by 26
  10. 2023Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimensioncited by 26
  11. 2022Posterior and Computational Uncertainty in Gaussian Processescited by 23
  12. 2021SigGPDE: Scaling Sparse Gaussian Processes on Sequential Datacited by 22
38 more
  1. 2023Koopman Kernel Regressioncited by 22
  2. 2023Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithmcited by 22
  3. 2021Uniform Generalization Bounds for Overparameterized Neural Networkscited by 21
  4. 2021Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networkscited by 20
  5. 2022AutoIP: A United Framework to Integrate Physics into Gaussian Processescited by 20
  6. 2023On the Optimality of Misspecified Kernel Ridge Regressioncited by 20
  7. 2021No-Regret Algorithms for Time-Varying Bayesian Optimizationcited by 19
  8. 2023On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decaycited by 19
  9. 2023Causal Modeling with Stationary Diffusionscited by 19
  10. 2022Improved Convergence Rates for Sparse Approximation Methods in Kernel-Based Learningcited by 18
  11. 2021Connections and Equivalences between the Nystr"om Method and Sparse Variational Gaussian Processescited by 17
  12. 2021Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applicationscited by 17
  13. 2021Bayesian Optimization over Permutation Spacescited by 17
  14. 2023Distributionally Robust Model-based Reinforcement Learning with Large State Spacescited by 17
  15. 2022Maximum Likelihood Estimation in Gaussian Process Regression is Ill-Posedcited by 16
  16. 2022Markovian Gaussian Process Variational Autoencoderscited by 16
  17. 2022What Can Be Learnt With Wide Convolutional Neural Networks?cited by 16
  18. 2023Random Exploration in Bayesian Optimization: Order-Optimal Regret and Computational Efficiencycited by 16
  19. 2021Bayesian Numerical Methods for Nonlinear Partial Differential Equationscited by 15
  20. 2021Fast Statistical Leverage Score Approximation in Kernel Ridge Regressioncited by 14
  21. 2021Robust Uncertainty Bounds in Reproducing Kernel Hilbert Spaces: A Convex Optimization Approachcited by 14
  22. 2021Learning curves for Gaussian process regression with power-law priors and targetscited by 14
  23. 2021Achieving Fairness with a Simple Ridge Penaltycited by 13
  24. 2021Open Problem: Tight Online Confidence Intervals for RKHS Elementscited by 13
  25. 2022Failure and success of the spectral bias prediction for Kernel Ridge Regression: the case of low-dimensional datacited by 12
  26. 2022Spectral Representation Learning for Conditional Moment Modelscited by 12
  27. 2022A Kernel Perspective of Skip Connections in Convolutional Networkscited by 12
  28. 2023Automatic nonlinear MPC approximation with closed-loop guaranteescited by 12
  29. 2024Computation-Aware Kalman Filtering and Smoothingcited by 12
  30. 2022A Robust Phased Elimination Algorithm for Corruption-Tolerant Gaussian Process Banditscited by 11
  31. 2023Linear Partial Monitoring for Sequential Decision-Making: Algorithms, Regret Bounds and Applicationscited by 11
  32. 2023Sample Path Regularity of Gaussian Processes from the Covariance Kernelcited by 11
  33. 2024FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learningcited by 11
  34. 2024Stein transport for Bayesian inferencecited by 11
  35. 2023Delayed Feedback in Kernel Banditscited by 10
  36. 2024Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Priorcited by 10
  37. 2021Structure Learning for Directed Treescited by 9
  38. 2024Data-Driven Distributionally Robust Safety Verification Using Barrier Certificates and Conditional Mean Embeddingscited by 9

50 most cited of 188 All 188 (JSON)