mlsanityData to 7 Oct 2026

On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime

arXiv 2005.13530 ↗First posted 27 May 2020math.AP cs.LG stat.MLcited by 38

Not on the map. Why?

Abstract (arXiv)

We describe a necessary and sufficient condition for the convergence to minimum Bayes risk when training two-layer ReLU-networks by gradient descent in the mean field regime with omni-directional initial parameter distribution. This article extends recent results of Chizat and Bach to ReLU-activated networks and to the situation in which there are no parameters which exactly achieve MBR. The condition does not depend on the initalization of parameters and concerns only the weak convergence of the realization of the neural network, not its parameter distribution.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 9; 2023: 8; 2024: 3; 2025: 5; 2026: 4.

38 papers.

Sort:

  1. 2022Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputscited by 74
  2. 2021Stochastic gradient descent with noise of machine learning type. Part II: Continuous time analysiscited by 25
  3. 2022Explicit Regularization in Overparametrized Models via Noise Injectioncited by 25
  4. 2021Overparameterization of deep ResNet: zero loss and mean-field analysiscited by 24
  5. 2022Infinite-width limit of deep linear neural networkscited by 20
  6. 2021Global Convergence of Three-layer Neural Networks in the Mean Field Regimecited by 16
  7. 2021Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalizationcited by 15
  8. 2021Analysis of feature learning in weight-tied autoencoders via the mean field lenscited by 13
  9. 2022On Feature Learning in Neural Networks with Global Convergence Guaranteescited by 12
  10. 2021Landscape analysis for shallow neural networks: complete classification of critical points for affine target functionscited by 11
  11. 2024Simplicity bias and optimization threshold in two-layer ReLU networkscited by 11
  12. 2021On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regimecited by 10
26 more
  1. 2021Generalization Error of GAN from the Discriminator's Perspectivecited by 8
  2. 2022On the symmetries in the dynamics of wide two-layer neural networkscited by 8
  3. 2024Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transportcited by 7
  4. 2022A blob method for inhomogeneous diffusion with applications to multi-agent control and samplingcited by 6
  5. 2023Rethinking Gauss-Newton for learning over-parameterized modelscited by 6
  6. 2023On the existence of minimizers in shallow residual ReLU neural network optimization landscapescited by 6
  7. 2021Limiting fluctuation and trajectorial stability of multilayer neural networks with mean field trainingcited by 5
  8. 2022Gradient descent provably escapes saddle points in the training of shallow ReLU networkscited by 5
  9. 2023Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Spacecited by 5
  10. 2025Optimal and Diffusion Transports in Machine Learningcited by 5
  11. 2023On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce itcited by 4
  12. 2022A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networkscited by 3
  13. 2023Minimum norm interpolation by perceptra: Explicit regularization and implicit biascited by 3
  14. 2024Generalization of Scaled Deep ResNets in the Mean-Field Regimecited by 3
  15. 2025Phase Diagram of Dropout for Two-Layer Neural Networks in the Mean-Field Regimecited by 3
  16. 2023Spectral Neural Networks: Approximation Theory and Optimization Landscapecited by 2
  17. 2025Convergence of Shallow ReLU Networks on Weakly Interacting Datacited by 2
  18. 2023Global Optimality of Elman-type RNN in the Mean-Field Regimecited by 1
  19. 2023Learning a Sparse Representation of Barron Functions with the Inverse Scale Space Flowcited by 1
  20. 2025Curse of Dimensionality in Neural Network Optimizationcited by 1
  21. 2026On the global convergence of gradient flow for wide shallow models beyond homogeneous nonlinearitiescited by 1
  22. 2026Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Methodcited by 1
  23. 2022Normalized gradient flow optimization in the training of ReLU artificial neural networkscited by 0
  24. 2025Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initializationcited by 0
  25. 2026Large Spikes in Stochastic Gradient Descent: A Large-Deviations Viewcited by 0
  26. 2026Singular perturbations and hierarchical learning in two-layer neural networkscited by 0