Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling
Not on the map. Why?
Abstract (arXiv)
We compute the asymptotic eigenvalue distribution of the neural tangent kernel of a two-layer neural network under a specific scaling of dimension. Namely, if X∈ℝn× d is an i.i.d random matrix, W∈ℝd× p is an i.i.d N(0,1) matrix and D∈ℝp× p is a diagonal matrix with i.i.d bounded entries, we consider the matrix [ NTK = frac1dXX^top odot frac1p σ'left( frac1√dXW right)D2 σ'left( frac1√dXW right)^top ] where σ' is a pseudo-Lipschitz function applied entrywise and under the scaling fracndp→ γ1 and fracpd→ γ2. We describe the asymptotic distribution as the free multiplicative convolution of the Marchenko--Pastur distribution with a deterministic distribution depending on σ and D.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 27 Aug 2025.
Builds on
32 of 52 references.
References · 32
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
- 2018On Lazy Training in Differentiable Programmingcited by 675
- 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
- 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
- 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
- 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
- 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 342
- 2019Two models of double descent for weak featurescited by 195
- 2020Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networkscited by 166
20 more
- 2020The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalizationcited by 101
- 2020The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Trainingcited by 84
- 2017A Random Matrix Approach to Neural Networkscited by 80
- 2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 77
- 2022Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regressioncited by 45
- 2023Deterministic equivalent and error universality of deep random features learningcited by 35
- 2021Hessian Eigenspectra of More Realistic Nonlinear Modelscited by 33
- 2019The generalization error of max-margin linear classifiers: Benign overfitting and high dimensional asymptotics in the overparametrized regimecited by 28
- 2019Eigenvalue distribution of nonlinear models of random matricescited by 27
- 2022An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices with Polynomial Scalingscited by 27
- 2024Nonlinear spiked covariance matrices and signal propagation in deep neural networkscited by 26
- 2024Asymptotics of Random Feature Regression Beyond the Linear Scaling Regimecited by 26
- 2021Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networkscited by 21
- 2023Six Lectures on Linearized Neural Networkscited by 20
- 2022Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networkscited by 13
- 2024Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regimecited by 12
- 2023Universality for the global spectrum of random inner-product kernel matrices in the polynomial regimecited by 10
- 2021Analysis of One-Hidden-Layer Neural Networks via the Resolvent Methodcited by 9
- 2023Deterministic equivalent of the Conjugate Kernel matrix associated to Artificial Neural Networkscited by 5
- 2025Global law of conjugate kernel random matrices with heavy-tailed weightscited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.