# Accelerating optimization over the space of probability measures

arXiv: https://arxiv.org/abs/2310.04006 · HTML: https://mlsanity.com/paper/2310.04006/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 84 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Builds on (most cited first, 30 of 38)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [A Mean Field View of the Landscape of Two-Layers Neural Networks](https://mlsanity.com/paper/1804.06561/) (2018)
- [On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport](https://mlsanity.com/paper/1805.09545/) (2018)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights](https://mlsanity.com/paper/1503.01243/) (2015)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [Underdamped Langevin MCMC: A non-asymptotic analysis](https://mlsanity.com/paper/1707.03663/) (2017)
- [Log-concave sampling: Metropolis-Hastings algorithms are fast](https://mlsanity.com/paper/1801.02309/) (2018)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [Convergence of Langevin MCMC in KL-divergence](https://mlsanity.com/paper/1705.09048/) (2017)
- [Variational inference via Wasserstein gradient flows](https://mlsanity.com/paper/2205.15902/) (2022)
- [On the geometry of Stein variational gradient descent](https://mlsanity.com/paper/1912.00894/) (2019)
- [The Randomized Midpoint Method for Log-Concave Sampling](https://mlsanity.com/paper/1909.05503/) (2019)
- [Understanding the Acceleration Phenomenon via High-Resolution Differential Equations](https://mlsanity.com/paper/1810.08907/) (2018)
- [Linear Coupling: An Ultimate Unification of Gradient and Mirror Descent](https://mlsanity.com/paper/1407.1537/) (2014)
- [Acceleration Methods](https://mlsanity.com/paper/2101.09545/) (2021)
- [Accelerating Langevin Sampling with Birth-death](https://mlsanity.com/paper/1905.09863/) (2019)
- [Accelerated Information Gradient flow](https://mlsanity.com/paper/1909.02102/) (2019)
- [A Dynamical Systems Perspective on Nesterov Acceleration](https://mlsanity.com/paper/1905.07436/) (2019)
- [Conformal Symplectic and Relativistic Optimization](https://mlsanity.com/paper/1903.04100/) (2019)
- [A Lyapunov Analysis of Momentum Methods in Optimization](https://mlsanity.com/paper/1611.02635/) (2016)
- [Improved Discretization Analysis for Underdamped Langevin Monte Carlo](https://mlsanity.com/paper/2302.08049/) (2023)
- [Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives](https://mlsanity.com/paper/2002.12493/) (2020)
- [Langevin Diffusion Variational Inference](https://mlsanity.com/paper/2208.07743/) (2022)
- [Overparameterization of deep ResNet: zero loss and mean-field analysis](https://mlsanity.com/paper/2105.14417/) (2021)

## Built on by (most cited first, 10 of 10)

- [Accelerated Stein Variational Gradient Flow](https://mlsanity.com/paper/2503.23462/) (2025)
- [A Particle Algorithm for Mean-Field Variational Inference](https://mlsanity.com/paper/2412.20385/) (2024)
- [Accelerated Markov Chain Monte Carlo Algorithms on Discrete States](https://mlsanity.com/paper/2505.12599/) (2025)
- [Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributions](https://mlsanity.com/paper/2509.04008/) (2025)
- [Sampling with Adaptive Variance for Multimodal Distributions](https://mlsanity.com/paper/2411.15220/) (2024)
- [Accelerated Regularized Wasserstein Proximal Sampling Algorithms](https://mlsanity.com/paper/2601.09848/) (2026)
- [AdamFlow: Adam-based Wasserstein Gradient Flows for Surface Registration in Medical Imaging](https://mlsanity.com/paper/2604.02290/) (2026)
- [Convex Hybrid Modeling: An Operator-Based Approach](https://mlsanity.com/paper/2605.23151/) (2026)
- [Nesterov acceleration in optimizing over probability measures](https://mlsanity.com/paper/2607.23008/) (2026)
- [Nonlocal Hamiltonian Dynamics on Sparse L'evy Graphs: Spectral Analysis and Multimodal Sampling](https://mlsanity.com/paper/2610.06904/) (2026)

## Abstract (arXiv)

The acceleration of gradient-based optimization methods is a subject of significant practical and theoretical importance, particularly within machine learning applications. While much attention has been directed towards optimizing within Euclidean space, the need to optimize over spaces of probability measures in machine learning motivates exploration of accelerated gradient methods in this context too. To this end, we introduce a Hamiltonian-flow approach analogous to momentum-based approaches in Euclidean space. We demonstrate that, in the continuous-time setting, algorithms based on this approach can achieve convergence rates of arbitrarily high order. We complement our findings with numerical examples.