mlsanityData to 7 Oct 2026

Accelerating optimization over the space of probability measures

arXiv 2310.04006 ↗First posted 6 Oct 2023math.OC cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

The acceleration of gradient-based optimization methods is a subject of significant practical and theoretical importance, particularly within machine learning applications. While much attention has been directed towards optimizing within Euclidean space, the need to optimize over spaces of probability measures in machine learning motivates exploration of accelerated gradient methods in this context too. To this end, we introduce a Hamiltonian-flow approach analogous to momentum-based approaches in Euclidean space. We demonstrate that, in the continuous-time setting, algorithms based on this approach can achieve convergence rates of arbitrarily high order. We complement our findings with numerical examples.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 38 of its 84 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

38 of 84 references.

References · 38

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2015Variational Inference with Normalizing Flowscited by 2,161
  5. 2016Variational Inference: A Review for Statisticianscited by 1,466
  6. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  7. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  8. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  9. 2022On Neural Differential Equationscited by 264
  10. 2015A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightscited by 203
  11. 2017Stein Variational Gradient Descent as Gradient Flowcited by 149
  12. 2017Underdamped Langevin MCMC: A non-asymptotic analysiscited by 131
26 more
  1. 2018Log-concave sampling: Metropolis-Hastings algorithms are fastcited by 117
  2. 2016A Variational Perspective on Accelerated Methods in Optimizationcited by 108
  3. 2017Convergence of Langevin MCMC in KL-divergencecited by 86
  4. 2022Variational inference via Wasserstein gradient flowscited by 84
  5. 2019On the geometry of Stein variational gradient descentcited by 82
  6. 2019The Randomized Midpoint Method for Log-Concave Samplingcited by 77
  7. 2018Understanding the Acceleration Phenomenon via High-Resolution Differential Equationscited by 68
  8. 2014Linear Coupling: An Ultimate Unification of Gradient and Mirror Descentcited by 58
  9. 2021Acceleration Methodscited by 48
  10. 2019Accelerating Langevin Sampling with Birth-deathcited by 45
  11. 2019Accelerated Information Gradient flowcited by 38
  12. 2019A Dynamical Systems Perspective on Nesterov Accelerationcited by 33
  13. 2016A Lyapunov Analysis of Momentum Methods in Optimizationcited by 28
  14. 2019Conformal Symplectic and Relativistic Optimizationcited by 28
  15. 2020Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectivescited by 27
  16. 2022Langevin Diffusion Variational Inferencecited by 26
  17. 2023Improved Discretization Analysis for Underdamped Langevin Monte Carlocited by 26
  18. 2021Overparameterization of deep ResNet: zero loss and mean-field analysiscited by 24
  19. 2018Direct Runge-Kutta Discretization Achieves Accelerationcited by 21
  20. 2019Accelerated Flow for Probability Distributionscited by 15
  21. 2019Generalized Momentum-Based Methods: A Hamiltonian Perspectivecited by 12
  22. 2021On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regimecited by 10
  23. 2022Second Order Ensemble Langevin Method for Sampling and Inverse Problemscited by 6
  24. 2015On Accelerated Methods in Optimizationcited by 5
  25. 2023Accelerated gradient descent method for functionals of probability measures by new convexity and smoothness based on transport mapscited by 3
  26. 2020Semi-discrete optimization through semi-discrete optimal transport: a framework for neural architecture searchcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 4.

9 papers.

  1. 2025Accelerated Stein Variational Gradient Flowcited by 5
  2. 2024A Particle Algorithm for Mean-Field Variational Inferencecited by 3
  3. 2025Towards understanding Accelerated Stein Variational Gradient Flow -- Analysis of Generalized Bilinear Kernels for Gaussian target distributionscited by 2
  4. 2024Sampling with Adaptive Variance for Multimodal Distributionscited by 1
  5. 2025Accelerated Markov Chain Monte Carlo Algorithms on Discrete Statescited by 0
  6. 2026Accelerated Regularized Wasserstein Proximal Sampling Algorithmscited by 0
  7. 2026AdamFlow: Adam-based Wasserstein Gradient Flows for Surface Registration in Medical Imagingcited by 0
  8. 2026Convex Hybrid Modeling: An Operator-Based Approachcited by 0
  9. 2026Nesterov acceleration in optimizing over probability measurescited by 0