mlsanityData to 7 Oct 2026

Kernel interpolation generalizes poorly

arXiv 2303.15809 ↗First posted 28 Mar 2023cs.LG math.ST stat.THcited by 16

Not on the map. Why?

Abstract (arXiv)

One of the most interesting problems in the recent renaissance of the studies in kernel regression might be whether the kernel interpolation can generalize well, since it may help us understand the `benign overfitting henomenon' reported in the literature on deep networks. In this paper, under mild conditions, we show that for any varepsilon>0, the generalization error of kernel interpolation is lower bounded by Omega(n-varepsilon). In other words, the kernel interpolation generalizes poorly for a large class of kernels. As a direct corollary, we can show that overfitted wide neural networks defined on the sphere generalize poorly.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 20 of its 35 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

20 of 35 references.

References · 20

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  3. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  4. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  5. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  6. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  7. 2019Benign Overfitting in Linear Regressioncited by 544
  8. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  9. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  10. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  11. 2019On the Inductive Bias of Neural Tangent Kernelscited by 177
  12. 2019Linearized two-layers neural networks in high dimensioncited by 155
8 more
  1. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
  2. 2020The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Trainingcited by 84
  3. 2020Deep Equals Shallow for ReLU Networks in Kernel Regimescited by 83
  4. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  5. 2018Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenoncited by 44
  6. 2023Generalization Ability of Wide Neural Networks on ℝcited by 28
  7. 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
  8. 2021The Three Stages of Learning Dynamics in High-Dimensional Kernel Methodscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 6; 2025: 1; 2026: 3.

16 papers.

Sort:

  1. 2023On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domainscited by 30
  2. 2023Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimensioncited by 26
  3. 2023On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decaycited by 19
  4. 2024Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrumcited by 13
  5. 2023Optimal Rate of Kernel Regression in Large Dimensionscited by 10
  6. 2024Optimal Rates of Kernel Ridge Regression under Source Condition in Large Dimensionscited by 9
  7. 2024The phase diagram of kernel interpolation in large dimensionscited by 9
  8. 2024Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decaycited by 8
  9. 2023Generalization Ability of Wide Residual Networkscited by 5
  10. 2023Changing the Kernel During Training Leads to Double Descent in Kernel Regressioncited by 1
  11. 2024On the Pinsker bound of inner product kernel regression in large dimensionscited by 1
  12. 2026On damage of interpolation to adversarial robustness in regressioncited by 1
4 more
  1. 2024Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theoriescited by 0
  2. 2025Sobolev norm inconsistency of kernel interpolationcited by 0
  3. 2026Harmful Overfitting in Sobolev Spacescited by 0
  4. 2026Large Dimensional Kernel Ridge Regression: Extending to Product Kernelscited by 0