mlsanityData to 7 Oct 2026

On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domains

arXiv 2305.02657 ↗First posted 4 May 2023stat.ML cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

In this paper, we provide a strategy to determine the eigenvalue decay rate (EDR) of a large class of kernel functions defined on a general domain rather than mathbb Sd. This class of kernel functions include but are not limited to the neural tangent kernel associated with neural networks with different depths and various activation functions. After proving that the dynamics of training the wide neural networks uniformly approximated that of the neural tangent kernel regression on general domains, we can further illustrate the minimax optimality of the wide neural network provided that the underground truth function f∈ [mathcal HNTK]s, an interpolation space associated with the RKHS HNTK of NTK. We also showed that the overfitted neural network can not generalize well. We believe our approach for determining the EDR of kernels might be also of independent interests.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 30 of its 55 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

30 of 55 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  5. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  6. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  7. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  8. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  9. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  10. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
  11. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  12. 2019Benign Overfitting in Linear Regressioncited by 544
18 more
  1. 2018Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Datacited by 260
  2. 2018Just Interpolate: Kernel "Ridgeless" Regression Can Generalizecited by 190
  3. 2019On the Inductive Bias of Neural Tangent Kernelscited by 177
  4. 2019The Convergence Rate of Neural Networks for Learned Functions of Different Frequenciescited by 134
  5. 2017Sobolev Norm Learning Rates for Regularized Least-Squares Algorithmcited by 113
  6. 2020Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networkscited by 87
  7. 2020The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Trainingcited by 84
  8. 2020Deep Equals Shallow for ReLU Networks in Kernel Regimescited by 83
  9. 2018On the Convergence Rate of Training Recurrent Neural Networkscited by 82
  10. 2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 76
  11. 2020On the Similarity between the Laplace and Neural Tangent Kernelscited by 76
  12. 2020Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHScited by 74
  13. 2022Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Datacited by 74
  14. 2018Optimal Rates for Spectral Algorithms with Least-Squares Regression over Hilbert Spacescited by 59
  15. 2018Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenoncited by 44
  16. 2020Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Networkcited by 38
  17. 2023Generalization Ability of Wide Neural Networks on ℝcited by 28
  18. 2023Kernel interpolation generalizes poorlycited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 11; 2026: 9.

30 papers.

Sort:

By area:

  1. 2023Optimal Rate of Kernel Regression in Large Dimensionscited by 10
  2. 2025Understanding Overadaptation in Supervised Fine-Tuning: The Role of Ensemble Methodscited by 9
  3. 2023Generalization Ability of Wide Residual Networkscited by 5
  4. 2024On the Impacts of the Random Initialization in the Neural Tangent Kernel Theorycited by 5
  5. 2025Neural Tangent Kernel of Neural Networks with Loss Informed by Differential Operatorscited by 3
  6. 2025Divergence of Empirical Neural Tangent Kernel in Classification Problemscited by 3
  7. 2025Learning Curves of Stochastic Gradient Descent in Kernel Regressioncited by 3
  8. 2025Beyond ReLU: How Activations Affect Neural Kernels and Random Wide Networkscited by 3
  9. 2026Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learningcited by 3
  10. 2024Improving Adaptivity via Over-Parameterization in Sequence Modelscited by 2
  11. 2024Gradient Descent Finds Over-Parameterized Neural Networks with Sharp Generalization for Nonparametric Regressioncited by 2
  12. 2025On the Saturation Effects of Spectral Algorithms in Large Dimensionscited by 2
18 more
  1. 2026MathAgent: Adversarial Evolution of Constraint Graphs for Mathematical Reasoning Data Synthesiscited by 2
  2. 2024On the Pinsker bound of inner product kernel regression in large dimensionscited by 1
  3. 2025A new initialisation to Control Gradients in Sinusoidal Neural networkcited by 1
  4. 2025Mathematical Foundations of Neural Tangents and Infinite-Width Networkscited by 1
  5. 2023Optimization dependent generalization bound for ReLU networks based on sensitivity in the tangent bundlecited by 0
  6. 2024Entrywise error bounds for low-rank approximations of kernel matricescited by 0
  7. 2024Mat'ern Kernels for Tunable Implicit Surface Reconstructioncited by 0
  8. 2024Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theoriescited by 0
  9. 2025Perturbing the Derivative: Wild Refitting for Model-Free Evaluation of Machine Learning Models under Bregman Lossescited by 0
  10. 2025Revisiting the Neural Tangent Kernel: the role of large width and depthcited by 0
  11. 2025Shallow Neural Networks Learn Low-Degree Spherical Polynomials with Feature Learning by Learnable Channel Attentioncited by 0
  12. 2026Pseudo-differential-enhanced physics-informed neural networkscited by 0
  13. 2026Optimal and Structure-Adaptive CATE Estimation with Kernel Ridge Regressioncited by 0
  14. 2026Gradient Descent with Projection Finds Over-Parameterized Neural Networks for Learning Low-Degree Polynomials with Nearly Minimax Optimal Ratecited by 0
  15. 2026Adversarial Robustness of NTK Neural Networkscited by 0
  16. 2026The Neural Tangent Kernel for Classificationcited by 0
  17. 2026The Differential Neural Tangent Kernel and Its Positivitycited by 0
  18. 2026DIAG: Diagnostic Iterative Alignment and Generation for Data-Efficient Mathematical Preference Distillationcited by 0