mlsanityData to 7 Oct 2026

A Comprehensive Analysis on the Learning Curve in Kernel Ridge Regression

arXiv 2410.17796 ↗First posted 23 Oct 2024cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper conducts a comprehensive study of the learning curves of kernel ridge regression (KRR) under minimal assumptions. Our contributions are three-fold: 1) we analyze the role of key properties of the kernel, such as its spectral eigen-decay, the characteristics of the eigenfunctions, and the smoothness of the kernel; 2) we demonstrate the validity of the Gaussian Equivalent Property (GEP), which states that the generalization performance of KRR remains the same when the whitened features are replaced by standard Gaussian vectors, thereby shedding light on the success of previous analyzes under the Gaussian Design Assumption; 3) we derive novel bounds that improve over existing bounds across a broad range of setting such as (in)dependent feature vectors and various combinations of eigen-decay rates in the over/underparameterized regimes.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 23 of its 51 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

23 of 51 references.

References · 23

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  3. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  4. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  5. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  6. 2020Benign overfitting in ridge regressioncited by 152
  7. 2020The Gaussian equivalence of generative models for learning with shallow neural networkscited by 113
  8. 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 95
  9. 2020Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimizationcited by 51
  10. 2023High-dimensional analysis of double descent for linear regression with random projectionscited by 47
  11. 2022Benign, Tempered, or Catastrophic: A Taxonomy of Overfittingcited by 36
  12. 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
11 more
  1. 2023Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimensioncited by 26
  2. 2023Generalization in Kernel Regression Under Realistic Assumptionscited by 26
  3. 2024A non-asymptotic theory of Kernel Ridge Regression: deterministic equivalents, test error, and GCV estimatorcited by 22
  4. 2023On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decaycited by 19
  5. 2024Characterizing Overfitting in Kernel Ridgeless Regression Through the Eigenspectrumcited by 13
  6. 2024On the Saturation Effect of Kernel Ridge Regressioncited by 12
  7. 2023Global Convergence of Deep Galerkin and PINNs Methods for Solving Partial Differential Equationscited by 10
  8. 2022Neural Q-learning for solving PDEscited by 8
  9. 2023A Theoretical Analysis of the Test Error of Finite-Rank Kernel Ridge Regressioncited by 8
  10. 2022Sobolev Spaces, Kernels and Discrepancies over Hyperspherescited by 5
  11. 2024Optimal Rates and Saturation for Noiseless Kernel Ridge Regressioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Beyond Benign Overfitting in Nadaraya-Watson Interpolatorscited by 2
  2. 2025Beyond Real Data: Synthetic Data through the Lens of Regularizationcited by 2
  3. 2025Beyond Universal Approximation Theorems: Algorithmic Uniform Approximation by Neural Networks Trained with Noisy Datacited by 1
  4. 2026Accelerating Regularized Attention Kernel Regression for Spectrum Cartographycited by 1
  5. 2025Spectral Analysis of Molecular Features: When Richer Features Do Not Guarantee Better Generalizationcited by 0
  6. 2026Benign Overfitting for General Norms and Distributionscited by 0