# Tight Worst-Case Bounds for the Smallest Eigenvalue of ReLU NTK Gram Matrices

arXiv: https://arxiv.org/abs/2608.03368 · HTML: https://mlsanity.com/paper/2608.03368/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 10 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 7 of 7)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Towards moderate overparameterization: global convergence guarantees for training shallow neural networks](https://mlsanity.com/paper/1902.04674/) (2019)
- [Effect of Activation Functions on the Training of Overparametrized Neural Nets](https://mlsanity.com/paper/1908.05660/) (2019)
- [Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension](https://mlsanity.com/paper/2405.14630/) (2024)
- [Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spheres](https://mlsanity.com/paper/2511.02625/) (2025)

## Built on by (most cited first, 2 of 2)

- [Training Under Challenge: Executable Certificates and Challenge-Closed Optimality for Neural Networks](https://mlsanity.com/paper/2608.12655/) (2026)
- [Dimension-Free Rank Lifting from Random Hyperplane Arrangements](https://mlsanity.com/paper/2609.39855/) (2026)

## Abstract (arXiv)

For n unit vectors x1,ldots,xn ∈ ℝd, we study the continuous ReLU derivative Gram matrix H, whose entries are obtained by averaging pairwise gated inner products over a standard Gaussian direction. Writing  Delta_± := mini neq j min |xi-xj|2, |xi+xj|2   for their projective separation, we prove the universal dimension-free lower bound  λmin(H) = Omega( Delta_±/√log n ) . Conversely, we construct worst-case families satisfying the matching upper bound  λmin(H) = O( Delta_±/√log n ) , showing that this rate is tight up to universal constants.