# Global law of conjugate kernel random matrices with heavy-tailed weights

arXiv: https://arxiv.org/abs/2502.18428 · HTML: https://mlsanity.com/paper/2502.18428/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Builds on (most cited first, 16 of 16)

- [Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning](https://mlsanity.com/paper/1810.01075/) (2018)
- [Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data](https://mlsanity.com/paper/2002.06716/) (2020)
- [A Random Matrix Approach to Neural Networks](https://mlsanity.com/paper/1702.05419/) (2017)
- [Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks](https://mlsanity.com/paper/2005.11879/) (2020)
- [Spectral Evolution and Invariance in Linear-width Neural Networks](https://mlsanity.com/paper/2211.06506/) (2022)
- [On the Spectrum of Random Features Maps of High Dimensional Data](https://mlsanity.com/paper/1805.11916/) (2018)
- [Eigenvalue distribution of nonlinear models of random matrices](https://mlsanity.com/paper/1904.03090/) (2019)
- [An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices with Polynomial Scalings](https://mlsanity.com/paper/2205.06308/) (2022)
- [Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks](https://mlsanity.com/paper/2109.09304/) (2021)
- [A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learning](https://mlsanity.com/paper/1912.00827/) (2019)
- [Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regime](https://mlsanity.com/paper/2408.01062/) (2024)
- [Universality for the global spectrum of random inner-product kernel matrices in the polynomial regime](https://mlsanity.com/paper/2310.18280/) (2023)
- [Analysis of One-Hidden-Layer Neural Networks via the Resolvent Method](https://mlsanity.com/paper/2105.05115/) (2021)
- [Deterministic equivalent of the Conjugate Kernel matrix associated to Artificial Neural Networks](https://mlsanity.com/paper/2306.05850/) (2023)
- [Eigenvalue Distribution of Large Random Matrices Arising in Deep Neural Networks: Orthogonal Case](https://mlsanity.com/paper/2201.04543/) (2022)
- [Entrywise application of non-linear functions on orthogonally invariant matrices](https://mlsanity.com/paper/2412.06943/) (2024)

## Built on by (most cited first, 3 of 3)

- [Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling](https://mlsanity.com/paper/2508.20036/) (2025)
- [Concentration of measure for non-linear random matrices with applications to neural networks and non-commutative polynomials](https://mlsanity.com/paper/2507.07625/) (2025)
- [A Sharp Universality Dichotomy for the Free Energy of Spherical Spin Glasses](https://mlsanity.com/paper/2601.08599/) (2026)

## Abstract (arXiv)

We study the asymptotic spectral distribution of the conjugate kernel random matrix YY^top, where Y= f(WX) arises from a two-layer neural network model. We consider the setting where W and X are random rectangular matrices with i.i.d. entries, where the entries of W follow a heavy-tailed distribution, while those of X have light tails. Our assumptions on W include a broad class of heavy-tailed distributions, such as symmetric α-stable laws with α ∈ ]0,2[ and sparse matrices with O(1) nonzero entries per row. The activation function f, applied entrywise, is bounded, smooth, odd, and nonlinear. We compute the limiting eigenvalue distribution of YY^top through its moments and show that heavy-tailed weights induce strong correlations between the entries of Y, resulting in richer and fundamentally different spectral behavior compared to the light-tailed case.