mlsanityData to 14 Oct 2026

Mathematical Foundations of Neural Tangents and Infinite-Width Networks

arXiv 2512.08264 ↗First posted 9 Dec 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

We investigate the mathematical foundations of neural networks in the infinite-width regime through the Neural Tangent Kernel (NTK). We propose the NTK-Eigenvalue-Controlled Residual Network (NTK-ECRN), an architecture integrating Fourier feature embeddings, residual connections with layerwise scaling, and stochastic depth to enable rigorous analysis of kernel evolution during training. Our theoretical contributions include deriving bounds on NTK dynamics, characterizing eigenvalue evolution, and linking spectral properties to generalization and optimization stability. Empirical results on synthetic and benchmark datasets validate the predicted kernel behavior and demonstrate improved training stability and generalization. This work provides a comprehensive framework bridging infinite-width theory and practical deep-learning architectures.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 11 of its 14 references that are in the map. First posted on arXiv 9 Dec 2025.

Builds on

11 of 14 references.

References · 11

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  2. 2020Feature Learning in Infinite-Width Neural Networkscited by 190
  3. 2020Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernelcited by 187
  4. 2020Tensor Programs II: Neural Tangent Kernel for Any Architecturecited by 143
  5. 2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 77
  6. 2023On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domainscited by 31
  7. 2019The asymptotic spectrum of the Hessian of DNN throughout trainingcited by 19
  8. 2021On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Modelscited by 13
  9. 2022On the Generalization Power of the Overfitted Three-Layer Neural Tangent Kernel Modelcited by 6
  10. 2025Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stabilitycited by 6
  11. 2025Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scalingcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025A Special Case of Quadratic Extrapolation Under the Neural Tangent Kernelcited by 0