mlsanityData to 7 Oct 2026

Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics

arXiv 2410.09697 ↗First posted 13 Oct 2024stat.ML cs.LG stat.COcited by 10

Not on the map. Why?

Abstract (arXiv)

Geometric tempering is a popular approach to sampling from challenging multi-modal probability distributions by instead sampling from a sequence of distributions which interpolate, using the geometric mean, between an easier proposal distribution and the target distribution. In this paper, we theoretically investigate the soundness of this approach when the sampling algorithm is Langevin dynamics, proving both upper and lower bounds. Our upper bounds are the first analysis in the literature under functional inequalities. They assert the convergence of tempered Langevin in continuous and discrete-time, and their minimization leads to closed-form optimal tempering schedules for some pairs of proposal and target distributions. Our lower bounds demonstrate a simple case where the geometric tempering takes exponential time, and further reveal that the geometric tempering can suffer from poor functional inequalities and slow convergence, even when the target distribution is well-conditioned. Overall, our results indicate that geometric tempering may not help, and can even be harmful for convergence.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 42 of its 86 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

42 of 86 references.

References · 42

  1. 2020Denoising Diffusion Implicit Modelscited by 9,655
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  7. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  8. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  9. 2019Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry Sufficescited by 224
  10. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  11. 2022Convergence for score-based generative modeling with polynomial complexitycited by 176
  12. 2020Stochastic Normalizing Flowscited by 156
30 more
  1. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 147
  2. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 144
  3. 2018Analysis of Langevin Monte Carlo via convex optimizationcited by 126
  4. 2018Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problemcited by 123
  5. 2020Structured Logconcave Sampling with a Restricted Gaussian Oraclecited by 85
  6. 2024Classifier-Free Guidance is a Predictor-Correctorcited by 79
  7. 2020Convergence of Langevin Monte Carlo in Chi-Squared and Renyi Divergencecited by 42
  8. 2023Reverse Diffusion Monte Carlocited by 41
  9. 2021Monte Carlo Variational Auto-Encoderscited by 36
  10. 2021Differentiable Annealed Importance Sampling and the Perils of Gradient Noisecited by 36
  11. 2020An invitation to sequential Monte Carlo samplerscited by 33
  12. 2024Sampling in Unit Time with Kernel Fisher-Rao Flowcited by 27
  13. 2024Stochastic Localization via Iterative Posterior Samplingcited by 27
  14. 2020A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributionscited by 26
  15. 2022Langevin Diffusion Variational Inferencecited by 26
  16. 2024Provable Benefit of Annealed Langevin Monte Carlo for Non-log-concave Samplingcited by 23
  17. 2024Implicit Diffusion: Efficient Optimization through Stochastic Samplingcited by 22
  18. 2024Zeroth-Order Sampling Methods for Non-Log-Concave Distributions: Alleviating Metastability by Denoising Diffusioncited by 19
  19. 2018Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decompositioncited by 16
  20. 2023An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient Flowcited by 16
  21. 2023A connection between Tempering and Entropic Mirror Descentcited by 16
  22. 2020Accelerating Nonconvex Learning via Replica Exchange Langevin Diffusioncited by 15
  23. 2023Efficient Training of Energy-Based Models Using Jarzynski Equalitycited by 15
  24. 2023Chain of Log-Concave Markov Chainscited by 13
  25. 2023Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Noise-Contrastive Estimation, and beyondcited by 13
  26. 2017Langevin Dynamics with Continuous Tempering for Training Deep Neural Networkscited by 7
  27. 2015Variational Optimization of Annealing Schedulescited by 5
  28. 2015Error Bounds for Sequential Monte Carlo Samplers for Multimodal Distributionscited by 4
  29. 2024Convergence Bounds for Sequential Monte Carlo on Multimodal Distributions using Soft Decompositioncited by 4
  30. 2022Finite Sample Complexity of Sequential Monte Carlo Estimators on Multimodal Target Distributionscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

  1. 2025Complexity Analysis of Normalizing Constant Estimation: from Jarzynski Equality to Annealed Importance Sampling and beyondcited by 11
  2. 2025Sequential Monte Carlo approximations of Wasserstein--Fisher--Rao gradient flowscited by 5
  3. 2025A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flowscited by 4
  4. 2026Diffusion Path Samplers via Sequential Monte Carlocited by 2
  5. 2025Restricted Spectral Gap Decomposition for Simulated Tempering Targeting Mixture Distributionscited by 1
  6. 2026Properties and limitations of geometric tempering for gradient flow dynamicscited by 0
  7. 2026Time-Inhomogeneous Preconditioned Langevin Dynamicscited by 0
  8. 2026Dimension-Uniform Discretization Analysis of Preconditioned Annealed Langevin Dynamics for Multimodal Gaussian Mixturescited by 0
  9. 2026Mode Coverage in Normalizing Flow Boltzmann Generators via Log-Ratio Variationcited by 0
  10. 2026R'enyi Tracking Bounds for Langevin Dynamics with Moving Targetscited by 0