# Collective evolution of weights in wide neural networks

arXiv: https://arxiv.org/abs/1810.03974 · HTML: https://mlsanity.com/paper/1810.03974/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2018.

## Built on by (most cited first, 2 of 2)

- [Universal scaling laws in the gradient descent training of neural networks](https://mlsanity.com/paper/2105.00507/) (2021)
- [Corner Gradient Descent](https://mlsanity.com/paper/2504.12519/) (2025)

## Abstract (arXiv)

We derive a nonlinear integro-differential transport equation describing collective evolution of weights under gradient descent in large-width neural-network-like models. We characterize stationary points of the evolution and analyze several scenarios where the transport equation can be solved approximately. We test our general method in the special case of linear free-knot splines, and find good agreement between theory and experiment in observations of global optima, stability of stationary points, and convergence rates.