mlsanityData to 7 Oct 2026

Convex Analysis of the Mean Field Langevin Dynamics

arXiv 2201.10469 ↗First posted 25 Jan 2022stat.ML cs.LG math.PRcited by 53

Not on the map. Why?

Abstract (arXiv)

As an example of the nonlinear Fokker-Planck equation, the mean field Langevin dynamics recently attracts attention due to its connection to (noisy) gradient descent on infinitely wide neural networks in the mean field regime, and hence the convergence property of the dynamics is of great theoretical interest. In this work, we give a concise and self-contained convergence rate analysis of the mean field Langevin dynamics with respect to the (regularized) objective function in both continuous and discrete time settings. The key ingredient of our proof is a proximal Gibbs distribution pq associated with the dynamics, which, in combination with techniques in [Vempala and Wibisono (2019)], allows us to develop a simple convergence theory parallel to classical results in convex optimization. Furthermore, we reveal that pq connects to the duality gap in the empirical risk minimization setting, which enables efficient empirical evaluation of the algorithm convergence.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 25 Jan 2022.

Builds on

29 of 50 references.

References · 29

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  3. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  4. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 224
  5. 2014Theoretical guarantees for approximate sampling from smooth and log-concave densitiescited by 206
  6. 2015Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithmcited by 195
  7. 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
  8. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  9. 2018Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernelcited by 131
  10. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  11. 2018Mean Field Analysis of Neural Networks: A Central Limit Theoremcited by 107
  12. 2017Stability and Generalization of Learning Algorithms that Converge to Global Optimacited by 100
17 more
  1. 2019Limitations of Lazy Training of Two-layers Neural Networkscited by 87
  2. 2020On the Convergence of Langevin Monte Carlo: The Interplay between Tail Growth and Smoothnesscited by 55
  3. 2017Stochastic Particle Gradient Descent for Infinite Ensemblescited by 52
  4. 2019Sparse Optimization on Measures with Over-parameterized Gradient Descentcited by 50
  5. 2019Mean-Field Langevin Dynamics and Energy Landscape of Neural Networkscited by 49
  6. 2020Convergence of Langevin Monte Carlo in Chi-Squared and Renyi Divergencecited by 42
  7. 2019Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyondcited by 33
  8. 2019Analysis of a Two-Layer Neural Network via Displacement Convexitycited by 27
  9. 2019Global convergence of neuron birth-death dynamicscited by 22
  10. 2020A Dynamical Central Limit Theorem for Shallow Neural Networkscited by 22
  11. 2020Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysiscited by 21
  12. 2021Convergence Rates of Gradient Methods for Convex Optimization in the Space of Measurescited by 19
  13. 2021Mixing Time Guarantees for Unadjusted Hamiltonian Monte Carlocited by 18
  14. 2019Mean-Field Neural ODEs via Relaxed Optimal Controlcited by 16
  15. 2021On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Settingcited by 16
  16. 2020Ergodicity of the underdamped mean-field Langevin dynamicscited by 7
  17. 2020Mirror Descent Algorithms for Minimizing Interacting Free Energycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 9; 2024: 13; 2025: 15; 2026: 7.

53 papers.

Sort:

By area:

  1. 2022High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representationcited by 146
  2. 2022Neural Networks Efficiently Learn Low-Dimensional Representations with SGDcited by 61
  3. 2022Learning threshold neurons via the "edge of stability"cited by 50
  4. 2024Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscapecited by 43
  5. 2023Gradient-Based Feature Learning under Structured Datacited by 32
  6. 2022Trajectory Inference via Mean-field Langevin in Path Spacecited by 29
  7. 2022Uniform-in-time propagation of chaos for mean field Langevin dynamicscited by 23
  8. 2023Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reductioncited by 19
  9. 2023Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimationcited by 17
  10. 2024Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamicscited by 14
  11. 2022Two-Scale Gradient Descent Ascent Dynamics Finds Mixed Nash Equilibria of Continuous Games: A Mean-Field Perspectivecited by 13
  12. 2024Partially Observed Trajectory Inference using Optimal Transport and a Dynamics Priorcited by 13
38 more
  1. 2023Momentum Particle Maximum Likelihoodcited by 12
  2. 2024Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspectivecited by 12
  3. 2024Improved Particle Approximation Error for Mean Field Neural Networkscited by 12
  4. 2023Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problemscited by 11
  5. 2023Doubly Regularized Entropic Wasserstein Barycenterscited by 10
  6. 2024Sampling from the Mean-Field Stationary Distributioncited by 7
  7. 2025Worst-case generation via minimax optimization in Wasserstein spacecited by 7
  8. 2025Scaling Diffusion Transformers Efficiently via μPcited by 6
  9. 2024Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samplescited by 5
  10. 2024Robust Feature Learning for Multi-Index Models in High Dimensionscited by 5
  11. 2025Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemblecited by 5
  12. 2023Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problemscited by 4
  13. 2025Direct Distributional Optimization for Provable Alignment of Diffusion Modelscited by 4
  14. 2025Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regimecited by 4
  15. 2022Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergencecited by 3
  16. 2022A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networkscited by 3
  17. 2024Linear convergence of proximal descent schemes on the Wasserstein spacecited by 3
  18. 2025Global Convergence and Rich Feature Learning in L-Layer Infinite-Width Neural Networks under μP Parametrizationcited by 3
  19. 2025Mirror Mean-Field Langevin Dynamicscited by 3
  20. 2022Mean-Field Analysis of Two-Layer Neural Networks: Global Optimality with Linear Convergence Ratescited by 2
  21. 2023Mean Field Optimization Problem Regularized by Fisher Informationcited by 2
  22. 2023Mean-field underdamped Langevin dynamics and its spacetime discretizationcited by 2
  23. 2024A unified Fourier slice method to derive ridgelet transform for a variety of depth-2 neural networkscited by 2
  24. 2024Symmetries in Overparametrized Neural Networks: A Mean-Field Viewcited by 2
  25. 2025Convergence of Time-Averaged Mean Field Gradient Descent Dynamics for Continuous Multi-Player Zero-Sum Gamescited by 2
  26. 2025Non-convex entropic mean-field optimization via Best Response flowcited by 2
  27. 2024Mirror Descent-Ascent for mean-field min-max problemscited by 1
  28. 2024Isoperimetry is All We Need: Langevin Posterior Sampling for RL with Sublinear Regretcited by 1
  29. 2025Convergence Analysis of the Wasserstein Proximal Algorithm beyond Geodesic Convexitycited by 1
  30. 2025Beyond Propagation of Chaos: A Stochastic Algorithm for Mean Field Optimizationcited by 1
  31. 2025Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Pointscited by 1
  32. 2026Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networkscited by 1
  33. 2026A note on convergence of Wasserstein policy optimizationcited by 1
  34. 2026Thinned Mean Field Langevin Dynamicscited by 1
  35. 2025Private Continuous-Time Synthetic Trajectory Generation via Mean-Field Langevin Dynamicscited by 0
  36. 2025Gaussian mixture layers for neural networkscited by 0
  37. 2025Feature Learning Dynamics in Infinite-Depth Neural Networkscited by 0
  38. 2026Functional Stochastic Localizationcited by 0

50 most cited of 53 All 53 (JSON)