# Universality for the global spectrum of random inner-product kernel matrices in the polynomial regime

arXiv: https://arxiv.org/abs/2310.18280 · HTML: https://mlsanity.com/paper/2310.18280/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 28 references that are in the map.
- First posted on arXiv 27 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [The generalization error of random features regression: Precise asymptotics and double descent curve](https://mlsanity.com/paper/1908.05355/) (2019)
- [Generalisation error in learning with random features and the hidden manifold model](https://mlsanity.com/paper/2002.09339/) (2020)
- [Learning curves of generic features maps for realistic datasets with a teacher-student model](https://mlsanity.com/paper/2102.08127/) (2021)
- [The Gaussian equivalence of generative models for learning with shallow neural networks](https://mlsanity.com/paper/2006.14709/) (2020)
- [A Random Matrix Approach to Neural Networks](https://mlsanity.com/paper/1702.05419/) (2017)
- [Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regression](https://mlsanity.com/paper/2204.10425/) (2022)
- [Eigenvalue distribution of nonlinear models of random matrices](https://mlsanity.com/paper/1904.03090/) (2019)
- [An Equivalence Principle for the Spectrum of Random Inner-Product Kernel Matrices with Polynomial Scalings](https://mlsanity.com/paper/2205.06308/) (2022)
- [Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networks](https://mlsanity.com/paper/2201.04753/) (2022)
- [Analysis of One-Hidden-Layer Neural Networks via the Resolvent Method](https://mlsanity.com/paper/2105.05115/) (2021)

## Built on by (most cited first, 10 of 10)

- [Asymptotic theory of in-context learning by linear attention](https://mlsanity.com/paper/2405.11751/) (2024)
- [Scaling and renormalization in high-dimensional regression](https://mlsanity.com/paper/2405.00592/) (2024)
- [Nonlinear spiked covariance matrices and signal propagation in deep neural networks](https://mlsanity.com/paper/2402.10127/) (2024)
- [The Nuclear Route: Sharp Asymptotics of ERM in Overparameterized Quadratic Networks](https://mlsanity.com/paper/2505.17958/) (2025)
- [A general technique for approximating high-dimensional empirical kernel matrices](https://mlsanity.com/paper/2511.03892/) (2025)
- [Global law of conjugate kernel random matrices with heavy-tailed weights](https://mlsanity.com/paper/2502.18428/) (2025)
- [Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scaling](https://mlsanity.com/paper/2508.20036/) (2025)
- [Multi-kernel spectral clustering: Entrywise eigenvector perturbation bounds and exact recovery](https://mlsanity.com/paper/2608.08704/) (2026)
- [Spectral graph clustering with inhomogeneous latent geometry](https://mlsanity.com/paper/2608.11321/) (2026)
- [Spectral Recovery of Point Clouds from Noisy Geometric Graphs](https://mlsanity.com/paper/2610.08634/) (2026)

## Abstract (arXiv)

We consider certain large random matrices, called random inner-product kernel matrices, which are essentially given by a nonlinear function f applied entrywise to a sample-covariance matrix, f(XTX), where X ∈ ℝd × N is random and normalized in such a way that f typically has order-one arguments. We work in the polynomial regime, where N asymp dℓ for some ℓ > 0, not just the linear regime where ℓ = 1. Earlier work by various authors showed that, when the columns of X are either uniform on the sphere or standard Gaussian vectors, and when ℓ is an integer (the linear regime ℓ = 1 is particularly well-studied), the bulk eigenvalues of such matrices behave in a simple way: They are asymptotically given by the free convolution of the semicircular and Marvcenko-Pastur distributions, with relative weights given by expanding f in the Hermite basis. In this paper, we show that this phenomenon is universal, holding as soon as X has i.i.d. entries with all finite moments. In the case of non-integer ℓ, the Marvcenko-Pastur term disappears (its weight in the free convolution vanishes), and the spectrum is just semicircular.