mlsanityData to 7 Oct 2026

Deterministic equivalent of the Conjugate Kernel matrix associated to Artificial Neural Networks

arXiv 2306.05850 ↗First posted 9 Jun 2023math.PR stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We study the Conjugate Kernel associated to a multi-layer linear-width feed-forward neural network with random weights, biases and data. We show that the empirical spectral distribution of the Conjugate Kernel converges to a deterministic limit. More precisely we obtain a deterministic equivalent for its Stieltjes transform and its resolvent, with quantitative bounds involving both the dimension and the spectral parameter. The limiting equivalent objects are described by iterating free convolution of measures and classical matrix operations involving the parameters of the model.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 11 of its 33 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

11 of 33 references.

References · 11

  1. 2017High-dimensional dynamics of generalization error in neural networkscited by 238
  2. 2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 76
  3. 2019A Fine-Grained Spectral Perspective on Neural Networkscited by 68
  4. 2020Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topologycited by 66
  5. 2023Deterministic equivalent and error universality of deep random features learningcited by 35
  6. 2019Eigenvalue distribution of nonlinear models of random matricescited by 27
  7. 2021Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networkscited by 21
  8. 2022Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networkscited by 12
  9. 2021Analysis of One-Hidden-Layer Neural Networks via the Resolvent Methodcited by 9
  10. 2022Quantitative deterministic equivalent of sample covariance matrices with a general dependence structurecited by 9
  11. 2021Resolvent convergence under heterogeneous second-moment profiles and quadratic-form controlcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 1.

5 papers.

  1. 2024Nonlinear spiked covariance matrices and signal propagation in deep neural networkscited by 25
  2. 2024Asymptotics of Learning with Deep Structured (Random) Featurescited by 13
  3. 2025Global law of conjugate kernel random matrices with heavy-tailed weightscited by 3
  4. 2025Eigenvalue distribution of the Neural Tangent Kernel in the quadratic scalingcited by 2
  5. 2026Bayesian Inference with Shaped Deep Non-linear MLPscited by 1