# Analysis of One-Hidden-Layer Neural Networks via the Resolvent Method

arXiv: https://arxiv.org/abs/2105.05115 · HTML: https://mlsanity.com/paper/2105.05115/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 11 May 2021.

## Builds on (most cited first, 5 of 5)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [A Random Matrix Approach to Neural Networks](https://mlsanity.com/paper/1702.05419/) (2017)
- [Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks](https://mlsanity.com/paper/2005.11879/) (2020)
- [On the Spectrum of Random Features Maps of High Dimensional Data](https://mlsanity.com/paper/1805.11916/) (2018)
- [A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning](https://mlsanity.com/paper/1912.00827/) (2019)

## Built on by (most cited first, 9 of 9)

- [Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization](https://mlsanity.com/paper/2205.10217/) (2022)
- [Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks](https://mlsanity.com/paper/2109.09304/) (2021)
- [Universality for the global spectrum of random inner-product kernel matrices in the polynomial regime](https://mlsanity.com/paper/2310.18280/) (2023)
- [Deterministic equivalent of the Conjugate Kernel matrix associated to Artificial Neural Networks](https://mlsanity.com/paper/2306.05850/) (2023)
- [Low Rank Gradients and Where to Find Them](https://mlsanity.com/paper/2510.01303/) (2025)
- [Global law of conjugate kernel random matrices with heavy-tailed weights](https://mlsanity.com/paper/2502.18428/) (2025)
- [Generalization for Least Squares Regression With Simple Spiked Covariances](https://mlsanity.com/paper/2410.13991/) (2024)
- [Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling](https://mlsanity.com/paper/2508.20036/) (2025)
- [Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix](https://mlsanity.com/paper/2510.06685/) (2025)

## Abstract (arXiv)

In this work, we investigate the asymptotic spectral density of the random feature matrix M = Y Y^ast with Y = f(WX) generated by a single-hidden-layer neural network, where W and X are random rectangular matrices with i.i.d. centred entries and f is a non-linear smooth function which is applied entry-wise. We prove that the Stieltjes transform of the limiting spectral distribution approximately satisfies a quartic self-consistent equation, which is exactly the equation obtained by [Pennington, Worah] and [Benigni, P'ech'e] with the moment method. We extend the previous results to the case of additive bias Y=f(WX+B) with B being an independent rank-one Gaussian random matrix, closer modelling the neural network infrastructures encountered in practice. Our key finding is that in the case of additive bias it is impossible to choose an activation function preserving the layer-to-layer singular value distribution, in sharp contrast to the bias-free case where a simple integral constraint is sufficient to achieve isospectrality. To obtain the asymptotics for the empirical spectral density we follow the resolvent method from random matrix theory via the cumulant expansion. We find that this approach is more robust and less combinatorial than the moment method and expect that it will apply also for models where the combinatorics of the former become intractable. The resolvent method has been widely employed, but compared to previous works, it is applied here to non-linear random matrices.