Data to 7 Oct 2026

Mean Field Analysis of Neural Networks: A Central Limit Theorem

arXiv 1808.09372 ↗First posted 28 Aug 2018math.PR math.ST stat.MLcited by 107

Not on the map. Why?

Abstract (arXiv)

We rigorously prove a central limit theorem for neural network models with a single hidden layer. The central limit theorem is proven in the asymptotic regime of simultaneously (A) large numbers of hidden units and (B) large numbers of stochastic gradient descent training iterations. Our result describes the neural network's fluctuations around its mean-field limit. The fluctuations have a Gaussian distribution and satisfy a stochastic partial differential equation. The proof relies upon weak convergence methods from stochastic analysis. In particular, we prove relative compactness for the sequence of processes and uniqueness of the limiting process in a suitable Sobolev space.

Key facts. Cited by 107 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 20; 2023: 20; 2024: 21; 2025: 15; 2026: 9.

107 papers.

Sort:

By area:

2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
2022Neural Networks can Learn Representations with Gradient Descentcited by 161
2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 86
2021Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstructioncited by 77
2022High-dimensional limit theorems for SGD: Effective dynamics and critical scalingcited by 69
2025Don't be lazy: CompleteP enables compute-efficient deep transformerscited by 69
2022Overview frequency principle/spectral bias in deep learningcited by 68
2021Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeedcited by 67
2023How Two-Layer Neural Networks Learn, One (Giant) Step at a Timecited by 67
2022Convex Analysis of the Mean Field Langevin Dynamicscited by 53
2022Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networkscited by 50
2024The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponentscited by 50

38 more

2021A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Networkcited by 47
2023A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networkscited by 46
2021Heavy Tails in SGD and Compressibility of Overparametrized Neural Networkscited by 44
2023From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networkscited by 42
2021KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Supportcited by 37
2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
2021Embedding Principle of Loss Landscape of Deep Neural Networkscited by 36
2024Asymptotics of feature learning in two-layer networks after one gradient-stepcited by 35
2024Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functionscited by 35
2021Towards Understanding the Condensation of Neural Networks at Initial Trainingcited by 31
2022Bayesian Interpolation with Deep Linear Networkscited by 21
2021Understanding Deflation Process in Over-parametrized Tensor Decompositioncited by 20
2023Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reductioncited by 19
2023Quantitative CLTs in Deep Neural Networkscited by 19
2023Six Lectures on Linearized Neural Networkscited by 19
2022Self-Consistency of the Fokker-Planck Equationcited by 18
2023Leveraging the two timescale regime to demonstrate convergence of neural networkscited by 18
2022Empirical Phase Diagram for Three-layer Neural Networks with Infinite Widthcited by 17
2021A theory of representation learning gives a deep generalisation of kernel methodscited by 15
2022An Analysis of Attention via the Lens of Exchangeability and Latent Variable Modelscited by 15
2023Phase Diagram of Initial Condensation for Two-layer Neural Networkscited by 15
2024Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizingcited by 15
2021Linear Frequency Principle Model to Understand the Absence of Overfitting in Neural Networkscited by 14
2022Neural parameter calibration for large-scale multi-agent modelscited by 14
2023Escaping mediocrity: how two-layer networks learn hard generalized linear models with SGDcited by 14
2024A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilitiescited by 14
2024On the Complexity of Learning Sparse Functions with Statistical and Gradient Queriescited by 13
2025An overview of condensation phenomenon in deep learningcited by 13
2023Fundamental limits of overparametrized shallow neural networks for supervised learningcited by 12
2024Improved Particle Approximation Error for Mean Field Neural Networkscited by 12
2021Implicit Bias of Linear RNNscited by 11
2024Coding schemes in neural networks learning classification taskscited by 11
2023Gaussian random field approximation via Stein's method with applications to wide random neural networkscited by 10
2021Redundant representations help generalization in wide neural networkscited by 9
2021Parallel Deep Neural Networks Have Zero Duality Gapcited by 9
2022Provably convergent quasistatic dynamics for mean-field two-player zero-sum gamescited by 9
2025Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networkscited by 9
2021On Energy-Based Models with Overparametrized Shallow Neural Networkscited by 8

50 most cited of 107 All 107 (JSON)