# Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction arXiv: https://arxiv.org/abs/2306.07221 ยท HTML: https://mlsanity.com/paper/2306.07221/ ## Key facts - Cited by 20 papers in the map (data through 14 Oct 2026). - Builds on 23 of its 51 references that are in the map. - First posted on arXiv 12 Jun 2023. ## Builds on (most cited first, 23 of 23) - [A Mean Field View of the Landscape of Two-Layers Neural Networks](https://mlsanity.com/paper/1804.06561/) (2018) - [On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport](https://mlsanity.com/paper/1805.09545/) (2018) - [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016) - [Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Suffices](https://mlsanity.com/paper/1903.08568/) (2019) - [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014) - [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015) - [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016) - [The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks](https://mlsanity.com/paper/2202.08658/) (2022) - [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019) - [Mean Field Analysis of Neural Networks: A Central Limit Theorem](https://mlsanity.com/paper/1808.09372/) (2018) - [Mean-Field Langevin Dynamics: Exponential Convergence and Annealing](https://mlsanity.com/paper/2202.01009/) (2022) - [Kernel Stein Discrepancy Descent](https://mlsanity.com/paper/2105.09994/) (2021) - [Convex Analysis of the Mean Field Langevin Dynamics](https://mlsanity.com/paper/2201.10469/) (2022) - [Stochastic Particle Gradient Descent for Infinite Ensembles](https://mlsanity.com/paper/1712.05438/) (2017) - [Mean-Field Langevin Dynamics and Energy Landscape of Neural Networks](https://mlsanity.com/paper/1905.07769/) (2019) - [Analysis of a Two-Layer Neural Network via Displacement Convexity](https://mlsanity.com/paper/1901.01375/) (2019) - [A Dynamical Central Limit Theorem for Shallow Neural Networks](https://mlsanity.com/paper/2008.09623/) (2020) - [Uniform-in-time propagation of chaos for mean field Langevin dynamics](https://mlsanity.com/paper/2212.03050/) (2022) - [Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysis](https://mlsanity.com/paper/2012.15477/) (2020) - [Quantitative Propagation of Chaos for SGD in Wide Neural Networks](https://mlsanity.com/paper/2007.06352/) (2020) - [Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization](https://mlsanity.com/paper/2203.16217/) (2022) - [Utilising the CLT Structure in Stochastic Gradient based Sampling : Improved Analysis and Faster Algorithms](https://mlsanity.com/paper/2206.03792/) (2022) - [Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems](https://mlsanity.com/paper/2303.02957/) (2023) ## Built on by (most cited first, 20 of 20) - [Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape](https://mlsanity.com/paper/2402.01258/) (2024) - [Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization](https://mlsanity.com/paper/2406.08654/) (2024) - [Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics](https://mlsanity.com/paper/2408.07254/) (2024) - [Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective](https://mlsanity.com/paper/2403.14917/) (2024) - [Momentum Particle Maximum Likelihood](https://mlsanity.com/paper/2312.07335/) (2023) - [Improved Particle Approximation Error for Mean Field Neural Networks](https://mlsanity.com/paper/2405.15767/) (2024) - [Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems](https://mlsanity.com/paper/2312.01127/) (2023) - [Sampling from the Mean-Field Stationary Distribution](https://mlsanity.com/paper/2402.07355/) (2024) - [Direct Distributional Optimization for Provable Alignment of Diffusion Models](https://mlsanity.com/paper/2502.02954/) (2025) - [Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble](https://mlsanity.com/paper/2502.05784/) (2025) - [Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks](https://mlsanity.com/paper/2605.22010/) (2026) - [Mean-field underdamped Langevin dynamics and its spacetime discretization](https://mlsanity.com/paper/2312.16360/) (2023) - [Symmetries in Overparametrized Neural Networks: A Mean-Field View](https://mlsanity.com/paper/2405.19995/) (2024) - [Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points](https://mlsanity.com/paper/2509.16974/) (2025) - [Thinned Mean Field Langevin Dynamics](https://mlsanity.com/paper/2605.28589/) (2026) - [Private Continuous-Time Synthetic Trajectory Generation via Mean-Field Langevin Dynamics](https://mlsanity.com/paper/2506.12203/) (2025) - [Local exponential stability of mean-field Langevin descent-ascent and associated particle system](https://mlsanity.com/paper/2602.01564/) (2026) - [Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine](https://mlsanity.com/paper/2605.20235/) (2026) - [Policy Gradient for Continuous-Time Robust Markov Decision Processes](https://mlsanity.com/paper/2606.04335/) (2026) - [On the Convergence of Semi-Gradient TD: A Mean Field Perspective](https://mlsanity.com/paper/2610.20014/) (2026) ## Abstract (arXiv) The mean-field Langevin dynamics (MFLD) is a nonlinear generalization of the Langevin dynamics that incorporates a distribution-dependent drift, and it naturally arises from the optimization of two-layer neural networks via (noisy) gradient descent. Recent works have shown that MFLD globally minimizes an entropy-regularized convex functional in the space of measures. However, all prior analyses assumed the infinite-particle or continuous-time limit, and cannot handle stochastic gradient updates. We provide an general framework to prove a uniform-in-time propagation of chaos for MFLD that takes into account the errors due to finite-particle approximation, time-discretization, and stochastic gradient approximation. To demonstrate the wide applicability of this framework, we establish quantitative convergence rate guarantees to the regularized global optimal solution under (i) a wide range of learning problems such as neural network in the mean-field regime and MMD minimization, and (ii) different gradient estimators including SGD and SVRG. Despite the generality of our results, we achieve an improved convergence rate in both the SGD and SVRG settings when specialized to the standard Langevin dynamics.