mlsanityData to 14 Oct 2026

Error estimates between SGD with momentum and underdamped Langevin diffusion

arXiv 2410.17297 ↗First posted 22 Oct 2024stat.ML cs.LG math.PRcited by 1

Not on the map. Why?

Abstract (arXiv)

Stochastic gradient descent with momentum is a popular variant of stochastic gradient descent, which has recently been reported to have a close relationship with the underdamped Langevin diffusion. In this paper, we establish a quantitative error estimate between them in the 1-Wasserstein and total variation distances.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 41 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

15 of 41 references.

References · 15

  1. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 243
  2. 2015Stochastic modified equations and adaptive stochastic gradient algorithmscited by 142
  3. 2017Underdamped Langevin MCMC: A non-asymptotic analysiscited by 133
  4. 2018Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundationscited by 114
  5. 2016Statistical Inference for Model Parameters in Stochastic Gradient Descentcited by 71
  6. 2018Sharp convergence rates for Langevin dynamics in the nonconvex settingcited by 68
  7. 2017Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methodscited by 49
  8. 2016On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integratorscited by 35
  9. 2019A Dynamical Systems Perspective on Nesterov Accelerationcited by 33
  10. 2018Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Accelerationcited by 31
  11. 2016Nesterov's Accelerated Gradient and Momentum as approximations to Regularised Update Descentcited by 26
  12. 2014Optimized first-order methods for smooth convex minimizationcited by 18
3 more
  1. 2015Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Settingcited by 10
  2. 2019Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Learningcited by 8
  3. 2020Understanding and Detecting Convergence for Stochastic Gradient Descent with Momentumcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Approximation to Deep Q-Network by Stochastic Delay Differential Equationscited by 0