mlsanityData to 7 Oct 2026

Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decay

arXiv 2401.01599 ↗First posted 3 Jan 2024cs.LG math.ST stat.THcited by 8

Not on the map. Why?

Abstract (arXiv)

The generalization error curve of certain kernel regression method aims at determining the exact order of generalization error with various source condition, noise level and choice of the regularization parameter rather than the minimax rate. In this work, under mild assumptions, we rigorously provide a full characterization of the generalization error curves of the kernel gradient descent method (and a large class of analytic spectral algorithms) in kernel regression. Consequently, we could sharpen the near inconsistency of kernel interpolation and clarify the saturation effects of kernel regression algorithms with higher qualification, etc. Thanks to the neural tangent kernel theory, these results greatly improve our understanding of the generalization behavior of training the wide neural networks. A novel technical contribution, the analytic functional argument, might be of independent interest.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 21 of its 42 references that are in the map. First posted on arXiv 3 Jan 2024.

Builds on

21 of 42 references.

References · 21

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  3. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  4. 2019Benign Overfitting in Linear Regressioncited by 544
  5. 2015An Introduction to Matrix Concentration Inequalitiescited by 391
  6. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  7. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  8. 2019Linearized two-layers neural networks in high dimensioncited by 155
  9. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
  10. 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 95
  11. 2016Optimal Rates For Regularization Of Statistical Inverse Learning Problemscited by 68
  12. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
9 more
  1. 2018Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenoncited by 44
  2. 2020Kernel regression in high dimensions: Refined analysis beyond double descentcited by 39
  3. 2023On the Optimality of Misspecified Spectral Algorithmscited by 28
  4. 2021The Three Stages of Learning Dynamics in High-Dimensional Kernel Methodscited by 21
  5. 2023On the Optimality of Misspecified Kernel Ridge Regressioncited by 20
  6. 2023On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decaycited by 19
  7. 2023Kernel interpolation generalizes poorlycited by 16
  8. 2023Optimal Rate of Kernel Regression in Large Dimensionscited by 10
  9. 2024Optimal Rates of Kernel Ridge Regression under Source Condition in Large Dimensionscited by 9

Built on by

Cited by year

Citing papers per year: 2024: 5; 2025: 3.

8 papers.

  1. 2024Optimal Rates for Vector-Valued Spectral Regularization Learning Algorithmscited by 12
  2. 2024On the Impacts of the Random Initialization in the Neural Tangent Kernel Theorycited by 5
  3. 2025Neural Tangent Kernel of Neural Networks with Loss Informed by Differential Operatorscited by 3
  4. 2024Improving Adaptivity via Over-Parameterization in Sequence Modelscited by 2
  5. 2025Fixed-Gaussian Spectral Algorithms: Minimax Optimal Rates for Misspecified Learning and Transfercited by 2
  6. 2025On the Saturation Effects of Spectral Algorithms in Large Dimensionscited by 2
  7. 2024On the Pinsker bound of inner product kernel regression in large dimensionscited by 1
  8. 2024Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theoriescited by 0