mlsanityData to 7 Oct 2026

Optimal Rates of Kernel Ridge Regression under Source Condition in Large Dimensions

arXiv 2401.01270 ↗First posted 2 Jan 2024cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Motivated by the studies of neural networks (e.g.,the neural tangent kernel theory), we perform a study on the large-dimensional behavior of kernel ridge regression (KRR) where the sample size n asymp dγ for some γ > 0. Given an RKHS H associated with an inner product kernel defined on the sphere mathbbSd, we suppose that the true function fρ* ∈ [H]s, the interpolation space of H with source condition s>0. We first determined the exact order (both upper and lower bound) of the generalization error of kernel ridge regression for the optimally chosen regularization parameter λ. We then further showed that when 0<s≤1, KRR is minimax optimal; and when s>1, KRR is not minimax optimal (a.k.a. he saturation effect). Our results illustrate that the curves of rate varying along γ exhibit the periodic plateau behavior and the multiple descent behavior and show how the curves evolve with s>0. Interestingly, our work provides a unified viewpoint of several recent works on kernel regression in the large-dimensional setting, which correspond to s=0 and s=1 respectively.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 17 of its 52 references that are in the map. First posted on arXiv 2 Jan 2024.

Builds on

17 of 52 references.

References · 17

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  3. 2020Benign overfitting in ridge regressioncited by 152
  4. 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 95
  5. 2022Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regressioncited by 42
  6. 2020Kernel regression in high dimensions: Refined analysis beyond double descentcited by 39
  7. 2021How rotational invariance of common kernels prevents generalization in high dimensionscited by 32
  8. 2023Generalization Ability of Wide Neural Networks on ℝcited by 28
  9. 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
  10. 2021The Three Stages of Learning Dynamics in High-Dimensional Kernel Methodscited by 21
  11. 2023On the Optimality of Misspecified Kernel Ridge Regressioncited by 20
  12. 2023On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decaycited by 19
5 more
  1. 2022Sharp Asymptotics of Kernel Ridge Regression Beyond the Linear Regimecited by 18
  2. 2023Kernel interpolation generalizes poorlycited by 16
  3. 2023Optimal Rate of Kernel Regression in Large Dimensionscited by 10
  4. 2023Strong inductive biases provably prevent harmless interpolationcited by 9
  5. 2019On the Multiple Descent of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 2.

9 papers.

  1. 2024The phase diagram of kernel interpolation in large dimensionscited by 9
  2. 2024Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decaycited by 8
  3. 2025Learning Curves of Stochastic Gradient Descent in Kernel Regressioncited by 3
  4. 2025Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfercited by 2
  5. 2025On the Saturation Effects of Spectral Algorithms in Large Dimensionscited by 2
  6. 2024On the Pinsker bound of inner product kernel regression in large dimensionscited by 1
  7. 2024Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theoriescited by 0
  8. 2026Optimal and Structure-Adaptive CATE Estimation with Kernel Ridge Regressioncited by 0
  9. 2026Large Dimensional Kernel Ridge Regression: Extending to Product Kernelscited by 0