# A Note on the Global Convergence of Multilayer Neural Networks in the Mean Field Regime

arXiv: https://arxiv.org/abs/2006.09355 · HTML: https://mlsanity.com/paper/2006.09355/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 4 of 4)

- [Global Convergence of Three-layer Neural Networks in the Mean Field Regime](https://mlsanity.com/paper/2105.05228/) (2021)
- [Analysis of feature learning in weight-tied autoencoders via the mean field lens](https://mlsanity.com/paper/2102.08373/) (2021)
- [Training Integrable Parameterizations of Deep Neural Networks in the Infinite-Width Limit](https://mlsanity.com/paper/2110.15596/) (2021)
- [Limiting fluctuation and trajectorial stability of multilayer neural networks with mean field training](https://mlsanity.com/paper/2110.15954/) (2021)

## Abstract (arXiv)

In a recent work, we introduced a rigorous framework to describe the mean field limit of the gradient-based learning dynamics of multilayer neural networks, based on the idea of a neuronal embedding. There we also proved a global convergence guarantee for three-layer (as well as two-layer) networks using this framework. In this companion note, we point out that the insights in our previous work can be readily extended to prove a global convergence guarantee for multilayer networks of any depths. Unlike our previous three-layer global convergence guarantee that assumes i.i.d. initializations, our present result applies to a type of correlated initialization. This initialization allows to, at any finite training time, propagate a certain universal approximation property through the depth of the neural network. To achieve this effect, we introduce a bidirectional diversity condition.