# Mathematical Foundations of Neural Tangents and Infinite-Width Networks

arXiv: https://arxiv.org/abs/2512.08264 · HTML: https://mlsanity.com/paper/2512.08264/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 14 references that are in the map.
- First posted on arXiv 9 Dec 2025.

## Builds on (most cited first, 11 of 11)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Feature Learning in Infinite-Width Neural Networks](https://mlsanity.com/paper/2011.14522/) (2020)
- [Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel](https://mlsanity.com/paper/2010.15110/) (2020)
- [Tensor Programs II: Neural Tangent Kernel for Any Architecture](https://mlsanity.com/paper/2006.14548/) (2020)
- [Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks](https://mlsanity.com/paper/2005.11879/) (2020)
- [On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domains](https://mlsanity.com/paper/2305.02657/) (2023)
- [The asymptotic spectrum of the Hessian of DNN throughout training](https://mlsanity.com/paper/1910.02875/) (2019)
- [On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models](https://mlsanity.com/paper/2103.05243/) (2021)
- [On the Generalization Power of the Overfitted Three-Layer Neural Tangent Kernel Model](https://mlsanity.com/paper/2206.02047/) (2022)
- [Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability](https://mlsanity.com/paper/2507.12837/) (2025)
- [Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling](https://mlsanity.com/paper/2508.20036/) (2025)

## Built on by (most cited first, 1 of 1)

- [A Special Case of Quadratic Extrapolation Under the Neural Tangent Kernel](https://mlsanity.com/paper/2512.15749/) (2025)

## Abstract (arXiv)

We investigate the mathematical foundations of neural networks in the infinite-width regime through the Neural Tangent Kernel (NTK). We propose the NTK-Eigenvalue-Controlled Residual Network (NTK-ECRN), an architecture integrating Fourier feature embeddings, residual connections with layerwise scaling, and stochastic depth to enable rigorous analysis of kernel evolution during training. Our theoretical contributions include deriving bounds on NTK dynamics, characterizing eigenvalue evolution, and linking spectral properties to generalization and optimization stability. Empirical results on synthetic and benchmark datasets validate the predicted kernel behavior and demonstrate improved training stability and generalization. This work provides a comprehensive framework bridging infinite-width theory and practical deep-learning architectures.