mlsanityData to 14 Oct 2026

The phase diagram of kernel interpolation in large dimensions

arXiv 2404.12597 ↗First posted 19 Apr 2024cs.LG math.ST stat.MLcited by 9

Not on the map. Why?

Abstract (arXiv)

The generalization ability of kernel interpolation in large dimensions (i.e., n asymp dγ for some γ>0) might be one of the most interesting problems in the recent renaissance of kernel regression, since it may help us understand the 'benign overfitting phenomenon' reported in the neural networks literature. Focusing on the inner product kernel on the sphere, we fully characterized the exact order of both the variance and bias of large-dimensional kernel interpolation under various source conditions s≥ 0. Consequently, we obtained the (s,γ)-phase diagram of large-dimensional kernel interpolation, i.e., we determined the regions in (s,γ)-plane where the kernel interpolation is minimax optimal, sub-optimal and inconsistent.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 19 Apr 2024.

Builds on

26 of 36 references.

References · 26

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  3. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  4. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  5. 2019Benign Overfitting in Linear Regressioncited by 549
  6. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  7. 2019Linearized two-layers neural networks in high dimensioncited by 155
  8. 2020Benign overfitting in ridge regressioncited by 153
  9. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 114
  10. 2021Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentrationcited by 110
  11. 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 96
  12. 2022Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regressioncited by 45
14 more
  1. 2018Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenoncited by 44
  2. 2022Benign, Tempered, or Catastrophic: A Taxonomy of Overfittingcited by 36
  3. 2021How rotational invariance of common kernels prevents generalization in high dimensionscited by 32
  4. 2023Generalization Ability of Wide Neural Networks on ℝcited by 30
  5. 2023Generalization in Kernel Regression Under Realistic Assumptionscited by 27
  6. 2023Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimensioncited by 26
  7. 2022Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regressioncited by 22
  8. 2022Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regimecited by 19
  9. 2023On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decaycited by 19
  10. 2023Kernel interpolation generalizes poorlycited by 16
  11. 2024Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrumcited by 13
  12. 2023Optimal Rate of Kernel Regression in Large Dimensionscited by 11
  13. 2024Optimal Rates of Kernel Ridge Regression under Source Condition in Large Dimensionscited by 11
  14. 2023Strong inductive biases provably prevent harmless interpolationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 1.

9 papers.

  1. 2024Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionalitycited by 7
  2. 2025Learning Curves of Stochastic Gradient Descent in Kernel Regressioncited by 4
  3. 2025When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMscited by 3
  4. 2024On the Pinsker bound of inner product kernel regression in large dimensionscited by 2
  5. 2025Beyond Benign Overfitting in Nadaraya-Watson Interpolatorscited by 2
  6. 2025On the Saturation Effects of Spectral Algorithms in Large Dimensionscited by 2
  7. 2025Benign Overfitting with Quantum Kernelscited by 2
  8. 2024Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theoriescited by 0
  9. 2026Large Dimensional Kernel Ridge Regression: Extending to Product Kernelscited by 0