# Mean-Field Langevin Dynamics: Exponential Convergence and Annealing

arXiv: https://arxiv.org/abs/2202.01009 · HTML: https://mlsanity.com/paper/2202.01009/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Feb 2022.

## Built on by (most cited first, 30 of 61)

- [High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation](https://mlsanity.com/paper/2205.01445/) (2022)
- [Neural Networks Efficiently Learn Low-Dimensional Representations with SGD](https://mlsanity.com/paper/2209.14863/) (2022)
- [Learning threshold neurons via the "edge of stability"](https://mlsanity.com/paper/2212.07469/) (2022)
- [Gradient-Based Feature Learning under Structured Data](https://mlsanity.com/paper/2309.03843/) (2023)
- [Trajectory Inference via Mean-field Langevin in Path Space](https://mlsanity.com/paper/2205.07146/) (2022)
- [Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics](https://mlsanity.com/paper/2211.00450/) (2022)
- [Uniform-in-time propagation of chaos for mean field Langevin dynamics](https://mlsanity.com/paper/2212.03050/) (2022)
- [Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction](https://mlsanity.com/paper/2306.07221/) (2023)
- [When does Metropolized Hamiltonian Monte Carlo provably outperform Metropolis-adjusted Langevin algorithm?](https://mlsanity.com/paper/2304.04724/) (2023)
- [Wasserstein Gradient Flows for Moreau Envelopes of f-Divergences in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2402.04613/) (2024)
- [Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics](https://mlsanity.com/paper/2408.07254/) (2024)
- [Two-Scale Gradient Descent Ascent Dynamics Finds Mixed Nash Equilibria of Continuous Games: A Mean-Field Perspective](https://mlsanity.com/paper/2212.08791/) (2022)
- [Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective](https://mlsanity.com/paper/2403.14917/) (2024)
- [Partially Observed Trajectory Inference using Optimal Transport and a Dynamics Prior](https://mlsanity.com/paper/2406.07475/) (2024)
- [Momentum Particle Maximum Likelihood](https://mlsanity.com/paper/2312.07335/) (2023)
- [Improved Particle Approximation Error for Mean Field Neural Networks](https://mlsanity.com/paper/2405.15767/) (2024)
- [Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems](https://mlsanity.com/paper/2312.01127/) (2023)
- [Doubly Regularized Entropic Wasserstein Barycenters](https://mlsanity.com/paper/2303.11844/) (2023)
- [Sampling from the Mean-Field Stationary Distribution](https://mlsanity.com/paper/2402.07355/) (2024)
- [Global Convergence of SGD On Two Layer Neural Nets](https://mlsanity.com/paper/2210.11452/) (2022)
- [Estimation Beyond Data Reweighting: Kernel Method of Moments](https://mlsanity.com/paper/2305.10898/) (2023)
- [Estimating the Rate-Distortion Function by Wasserstein Gradient Descent](https://mlsanity.com/paper/2310.18908/) (2023)
- [Direct Distributional Optimization for Provable Alignment of Diffusion Models](https://mlsanity.com/paper/2502.02954/) (2025)
- [Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble](https://mlsanity.com/paper/2502.05784/) (2025)
- [Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime](https://mlsanity.com/paper/2504.18208/) (2025)
- [Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems](https://mlsanity.com/paper/2303.02957/) (2023)
- [Inclusive KL Gradient Flows: Otto-Wasserstein, Fisher-Rao-Gaussian, and Local-Estimator Dynamics](https://mlsanity.com/paper/2411.00214/) (2024)
- [Linear convergence of proximal descent schemes on the Wasserstein space](https://mlsanity.com/paper/2411.15067/) (2024)
- [Mean-Field Analysis of Two-Layer Neural Networks: Global Optimality with Linear Convergence Rates](https://mlsanity.com/paper/2205.09860/) (2022)
- [Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence](https://mlsanity.com/paper/2210.06819/) (2022)

## Abstract (arXiv)

Noisy particle gradient descent (NPGD) is an algorithm to minimize convex functions over the space of measures that include an entropy term. In the many-particle limit, this algorithm is described by a Mean-Field Langevin dynamics - a generalization of the Langevin dynamics with a non-linear drift - which is our main object of study. Previous work have shown its convergence to the unique minimizer via non-quantitative arguments. We prove that this dynamics converges at an exponential rate, under the assumption that a certain family of Log-Sobolev inequalities holds. This assumption holds for instance for the minimization of the risk of certain two-layer neural networks, where NPGD is equivalent to standard noisy gradient descent. We also study the annealed dynamics, and show that for a noise decaying at a logarithmic rate, the dynamics converges in value to the global minimizer of the unregularized objective function.