Tight Worst-Case Bounds for the Smallest Eigenvalue of ReLU NTK Gram Matrices
Not on the map. Why?
Abstract (arXiv)
For n unit vectors x1,ldots,xn ∈ ℝd, we study the continuous ReLU derivative Gram matrix H, whose entries are obtained by averaging pairwise gated inner products over a standard Gaussian direction. Writing Delta_± := mini neq j min |xi-xj|2, |xi+xj|2 for their projective separation, we prove the universal dimension-free lower bound λmin(H) = Omega( Delta_±/√log n ) . Conversely, we construct worst-case families satisfying the matching upper bound λmin(H) = O( Delta_±/√log n ) , showing that this rate is tight up to universal constants.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 7 of its 10 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
7 of 10 references.
References · 7
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
- 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
- 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
- 2019Effect of Activation Functions on the Training of Overparametrized Neural Netscited by 14
- 2024Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimensioncited by 13
- 2025Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spherescited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.