mlsanityData to 14 Oct 2026

Global law of conjugate kernel random matrices with heavy-tailed weights

arXiv 2502.18428 ↗First posted 25 Feb 2025math.PR cs.LG stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

We study the asymptotic spectral distribution of the conjugate kernel random matrix YY^top, where Y= f(WX) arises from a two-layer neural network model. We consider the setting where W and X are random rectangular matrices with i.i.d. entries, where the entries of W follow a heavy-tailed distribution, while those of X have light tails. Our assumptions on W include a broad class of heavy-tailed distributions, such as symmetric α-stable laws with α ∈ ]0,2[ and sparse matrices with O(1) nonzero entries per row. The activation function f, applied entrywise, is bounded, smooth, odd, and nonlinear. We compute the limiting eigenvalue distribution of YY^top through its moments and show that heavy-tailed weights induce strong correlations between the entries of Y, resulting in richer and fundamentally different spectral behavior compared to the light-tailed case.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 16 of its 38 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

16 of 38 references.

References · 16

  1. 2018Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learningcited by 215
  2. 2020Predicting trends in the quality of state-of-the-art neural networks without access to training or testing datacited by 115
  3. 2017A Random Matrix Approach to Neural Networkscited by 80
  4. 2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 77
  5. 2022Spectral Evolution and Invariance in Linear-width Neural Networkscited by 30
  6. 2018On the Spectrum of Random Features Maps of High Dimensional Datacited by 28
  7. 2019Eigenvalue distribution of nonlinear models of random matricescited by 27
  8. 2022An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices with Polynomial Scalingscited by 27
  9. 2021Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networkscited by 21
  10. 2019A Random Matrix Perspective on Mixtures of Nonlinearities for Deep Learningcited by 20
  11. 2024Universality of Kernel Random Matrices and Kernel Regression in the Quadratic Regimecited by 12
  12. 2023Universality for the global spectrum of random inner-product kernel matrices in the polynomial regimecited by 10
4 more
  1. 2021Analysis of One-Hidden-Layer Neural Networks via the Resolvent Methodcited by 9
  2. 2023Deterministic equivalent of the Conjugate Kernel matrix associated to Artificial Neural Networkscited by 5
  3. 2022Eigenvalue Distribution of Large Random Matrices Arising in Deep Neural Networks: Orthogonal Casecited by 4
  4. 2024Entrywise application of non-linear functions on orthogonally invariant matricescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scalingcited by 2
  2. 2025Concentration of measure for non-linear random matrices with applications to neural networks and non-commutative polynomialscited by 0
  3. 2026A Sharp Universality Dichotomy for the Free Energy of Spherical Spin Glassescited by 0