# Genus expansion for non-linear random matrix ensembles with applications to neural networks

arXiv: https://arxiv.org/abs/2407.08459 · HTML: https://mlsanity.com/paper/2407.08459/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2024.

## Built on by (most cited first, 4 of 4)

- [Fixed-Point RNNs: Interpolating from Diagonal to Dense](https://mlsanity.com/paper/2503.10799/) (2025)
- [Universality of first-order methods on random and deterministic matrices](https://mlsanity.com/paper/2604.11729/) (2026)
- [Advances in Neural Controlled Differential Equations](https://mlsanity.com/paper/2607.05280/) (2026)
- [K"ahler landscapes for complex neural network descents and guarantees including a search and destroy of the Calabi-Yau manifold](https://mlsanity.com/paper/2608.19584/) (2026)

## Abstract (arXiv)

We present a unified approach to studying certain non-linear random matrix ensembles and associated random neural networks at initialization. This begins with a novel series expansion for neural networks which generalizes Fa'a di Bruno's formula to an arbitrary number of compositions. The role of monomials is played by random multilinear maps indexed by directed graphs, whose edges correspond to random matrices. Crucially, this expansion linearizes the effect of the activation functions, allowing for the direct application of Wick's principle and the genus expansion technique. As an application, we prove several results about neural networks with random weights. We first give a new proof of the fact that they converge to Gaussian processes as their width tends to infinity. Secondly, we quantify the rate of convergence of the Neural Tangent Kernel to its deterministic limit in Frobenius norm. Finally, we compute the moments of the limiting spectral distribution of the Jacobian (only the first two of which were previously known), expressing them as sums over non-crossing partitions. All of these results are then generalised to the case of neural networks with sparse and non-Gaussian weights, under moment assumptions.