mlsanityData to 13 Oct 2026

Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization

arXiv 1707.06618 ↗First posted 20 Jul 2017stat.ML cs.LG math.OCcited by 97

Not on the map. Why?

Abstract (arXiv)

We present a unified framework to analyze the global convergence of Langevin dynamics based algorithms for nonconvex finite-sum optimization with n component functions. At the core of our analysis is a direct analysis of the ergodicity of the numerical approximations to Langevin dynamics, which leads to faster convergence rates. Specifically, we show that gradient Langevin dynamics (GLD) and stochastic gradient Langevin dynamics (SGLD) converge to the almost minimizer within tilde Obig(nd/(λε) big) and tilde Obig(d7/(λ5ε5) big) stochastic gradient evaluations respectively, where d is the problem dimension, and λ is the spectral gap of the Markov chain generated by GLD. Both results improve upon the best known gradient complexity results (Raginsky et al., 2017). Furthermore, for the first time we prove the global convergence guarantee for variance reduced stochastic gradient Langevin dynamics (SVRG-LD) to the almost minimizer within tilde Obig(√nd5/(λ4ε5/2)big) stochastic gradient evaluations, which outperforms the gradient complexities of GLD and SGLD in a wide regime. Our theoretical analyses shed some light on using Langevin dynamics based algorithms for nonconvex optimization with provable guarantees.

Key facts. Cited by 97 papers in the map (data through 13 Oct 2026). First posted on arXiv 20 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 10; 2022: 19; 2023: 23; 2024: 12; 2025: 19; 2026: 14.

97 papers.

Sort:

By area:

  1. 2024Constrained Synthesis with Projected Diffusion Modelscited by 86
  2. 2022Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlocited by 63
  3. 2025Simultaneous Multi-Robot Motion Planning with Projected Diffusion Modelscited by 37
  4. 2024Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Datacited by 36
  5. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 34
  6. 2023Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlocited by 29
  7. 2021Time-independent Generalization Bounds for SGLD in Non-convex Settingscited by 27
  8. 2022Langevin Monte Carlo for Contextual Banditscited by 22
  9. 2025Training-Free Constrained Generation With Stable Diffusion Modelscited by 20
  10. 2023Guiding Energy-based Models via Contrastive Latent Variablescited by 17
  11. 2022Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimizationcited by 16
  12. 2025Constrained Discrete Diffusioncited by 16
38 more
  1. 2021Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channelscited by 15
  2. 2022Fisher information lower bounds for samplingcited by 12
  3. 2023Time-Independent Information-Theoretic Generalization Bounds for SGLDcited by 12
  4. 2024More Efficient Randomized Exploration for Reinforcement Learning via Approximate Samplingcited by 12
  5. 2021Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networkscited by 11
  6. 2022Score-Guided Intermediate Layer Optimization: Fast Langevin Mixing for Inverse Problemscited by 11
  7. 2021Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation functioncited by 10
  8. 2024Accelerating Approximate Thompson Sampling with Underdamped Langevin Monte Carlocited by 10
  9. 2025Neuro-Symbolic Generative Diffusion Models for Physically Grounded, Robust, and Safe Generationcited by 10
  10. 2022Interacting Contour Stochastic Gradient Langevin Dynamicscited by 9
  11. 2023Convergence and concentration properties of constant step-size SGD through Markov chainscited by 9
  12. 2024Energy-based Hopfield Boosting for Out-of-Distribution Detectioncited by 9
  13. 2021Stochastic Gradient Langevin Dynamics with Variance Reductioncited by 7
  14. 2022Non-reversible Parallel Tempering for Deep Posterior Approximationcited by 7
  15. 2025Hardness of Learning Fixed Parities with Neural Networkscited by 7
  16. 2026Parallel Stochastic Gradient-Based Planning for World Modelscited by 7
  17. 2021Stochastic Anderson Mixing for Nonconvex Stochastic Optimizationcited by 6
  18. 2022Constrained Langevin Algorithms with L-mixing External Random Variablescited by 6
  19. 2022Langevin dynamics based algorithm e-THvarepsilonO POULA for stochastic optimization problems with discontinuous stochastic gradientcited by 5
  20. 2024Faster Sampling via Stochastic Gradient Proximal Samplercited by 5
  21. 2025From Global to Local: A Scalable Benchmark for Local Posterior Samplingcited by 5
  22. 2021A Unifying and Canonical Description of Measure-Preserving Diffusionscited by 4
  23. 2023Non-Log-Concave and Nonsmooth Sampling via Langevin Monte Carlo Algorithmscited by 4
  24. 2023Constructing Semantics-Aware Adversarial Examples with a Probabilistic Perspectivecited by 4
  25. 2023Langevin Quasi-Monte Carlocited by 4
  26. 2025Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdowncited by 4
  27. 2021Revisiting the Effects of Stochasticity for Hamiltonian Samplerscited by 3
  28. 2021Unadjusted Langevin algorithm for sampling a mixture of weakly smooth potentialscited by 3
  29. 2022Convergence Error Analysis of Reflected Gradient Langevin Dynamics for Globally Optimizing Non-Convex Constrained Problemscited by 3
  30. 2023Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descentcited by 3
  31. 2023Generalization Bounds for Label Noise Stochastic Gradient Descentcited by 3
  32. 2025kTULA: A Langevin sampling algorithm with improved KL bounds under super-linear log-gradientscited by 3
  33. 2022Global convergence of optimized adaptive importance samplerscited by 2
  34. 2022Weak Convergence of Approximate reflection coupling and its Application to Non-convex Optimizationcited by 2
  35. 2022A Particle-Based Algorithm for Distributional Optimization on textitConstrained Domains via Variational Transport and Mirror Descentcited by 2
  36. 2022A Dynamical System View of Langevin-Based Non-Convex Samplingcited by 2
  37. 2023Gradient-based Wang-Landau Algorithm: A Novel Sampler for Output Distribution of Neural Networks over the Input Spacecited by 2
  38. 2024Fast Value Tracking for Deep Reinforcement Learningcited by 2

50 most cited of 97 All 97 (JSON)