mlsanityData to 7 Oct 2026

Kernel Interpolation as a Bayes Point Machine

arXiv 2110.04274 ↗First posted 8 Oct 2021cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

A Bayes point machine is a single classifier that approximates the majority decision of an ensemble of classifiers. This paper observes that kernel interpolation is a Bayes point machine for Gaussian process classification. This observation facilitates the transfer of results from both ensemble theory as well as an area of convex geometry known as Brunn-Minkowski theory to derive PAC-Bayes risk bounds for kernel interpolation. Since large margin, infinite width neural networks are kernel interpolators, the paper's findings may help to explain generalisation in neural networks more broadly. Supporting this idea, the paper finds evidence that large margin, finite width neural networks behave like Bayes point machines too.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 33 of its 76 references that are in the map. First posted on arXiv 8 Oct 2021.

Builds on

33 of 76 references.

References · 33

  1. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2019When Does Label Smoothing Help?cited by 985
  4. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  5. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  6. 2017Deep Neural Networks as Gaussian Processescited by 617
  7. 2019Benign Overfitting in Linear Regressioncited by 544
  8. 2018Born Again Neural Networkscited by 539
  9. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  10. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  11. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  12. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
21 more
  1. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  2. 2018Gaussian Processes and Kernel Methods: A Review on Connections and Equivalencescited by 188
  3. 2018To understand deep learning we need to understand kernel learningcited by 186
  4. 2021On the Origin of Implicit Regularization in Stochastic Gradient Descentcited by 183
  5. 2020Self-Distillation Amplifies Regularization in Hilbert Spacecited by 171
  6. 2021A Universal Law of Robustness via Isoperimetrycited by 150
  7. 2020Implicit Gradient Regularizationcited by 145
  8. 2016Toward Deeper Understanding of Neural Networks: The Power of Initialization and a Dual View on Expressivitycited by 138
  9. 2020Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Taskscited by 137
  10. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  11. 2020Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regimecited by 81
  12. 2020Self-Distillation as Instance-Specific Label Smoothingcited by 71
  13. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
  14. 2018Learning Invariances using the Marginal Likelihoodcited by 51
  15. 2015Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithmcited by 50
  16. 2020Generalization bounds for deep learningcited by 38
  17. 2020Second Order PAC-Bayesian Bounds for the Weighted Majority Votecited by 34
  18. 2021Learning by Turning: Neural Architecture Aware Optimisationcited by 23
  19. 2021Interpolating Classifiers Make Few Mistakescited by 22
  20. 2021A General Framework for the Practical Disintegration of PAC-Bayesian Boundscited by 21
  21. 2021On Margins and Derandomisation in PAC-Bayescited by 19

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Pathologies of Predictive Diversity in Deep Ensemblescited by 16