# Scaling Limit of Neural Networks with the Xavier Initialization and Convergence to a Global Minimum

arXiv: https://arxiv.org/abs/1907.04108 · HTML: https://mlsanity.com/paper/1907.04108/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2019.

## Built on by (most cited first, 3 of 3)

- [GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing](https://mlsanity.com/paper/2505.04915/) (2025)
- [Minimum norm interpolation by perceptra: Explicit regularization and implicit bias](https://mlsanity.com/paper/2311.06138/) (2023)
- [Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit](https://mlsanity.com/paper/2605.08352/) (2026)

## Abstract (arXiv)

We analyze single-layer neural networks with the Xavier initialization in the asymptotic regime of large numbers of hidden units and large numbers of stochastic gradient descent training steps. The evolution of the neural network during training can be viewed as a stochastic system and, using techniques from stochastic analysis, we prove the neural network converges in distribution to a random ODE with a Gaussian distribution. The limit is completely different than in the typical mean-field results for neural networks due to the frac1√N normalization factor in the Xavier initialization (versus the frac1N factor in the typical mean-field framework). Although the pre-limit problem of optimizing a neural network is non-convex (and therefore the neural network may converge to a local minimum), the limit equation minimizes a (quadratic) convex objective function and therefore converges to a global minimum. Furthermore, under reasonable assumptions, the matrix in the limiting quadratic objective function is positive definite and thus the neural network (in the limit) will converge to a global minimum with zero loss on the training set.