mlsanityData to 7 Oct 2026

Stochastic Gradient Langevin Dynamics with Variance Reduction

arXiv 2102.06759 ↗First posted 12 Feb 2021cs.LG math.OCcited by 7

Not on the map. Why?

Abstract (arXiv)

Stochastic gradient Langevin dynamics (SGLD) has gained the attention of optimization researchers due to its global optimization properties. This paper proves an improved convergence property to local minimizers of nonconvex objective functions using SGLD accelerated by variance reductions. Moreover, we prove an ergodicity property of the SGLD scheme, which gives insights on its potential to find global minimizers of nonconvex objectives.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 20 of its 38 references that are in the map. First posted on arXiv 12 Feb 2021.

Builds on

20 of 38 references.

References · 20

  1. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  2. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 401
  3. 2017How to Escape Saddle Points Efficientlycited by 284
  4. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  5. 2015Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decompositioncited by 241
  6. 2016Stochastic Variance Reduction for Nonconvex Optimizationcited by 143
  7. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  8. 2018An Alternative View: When Does SGD Escape Local Minima?cited by 115
  9. 2017Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimizationcited by 96
  10. 2017Convergence of Langevin MCMC in KL-divergencecited by 86
  11. 2016Variance Reduction for Faster Non-Convex Optimizationcited by 72
  12. 2017A Hitting Time Analysis of Stochastic Gradient Langevin Dynamicscited by 70
8 more
  1. 2019SSRGD: Simple Stochastic Recursive Gradient Descent for Escaping Saddle Pointscited by 25
  2. 2018Escaping Saddle Points in Constrained Optimizationcited by 14
  3. 2019On Stationary-Point Hitting Time and Ergodicity of Stochastic Gradient Langevin Dynamicscited by 14
  4. 2019Heavy-ball Algorithms Always Escape Saddle Pointscited by 13
  5. 2019Efficiently avoiding saddle points with zero order methods: No gradients requiredcited by 10
  6. 2020Replica Exchange for Non-Convex Optimizationcited by 10
  7. 2019Perturbed Proximal Descent to Escape Saddle Points for Non-convex and Non-smooth Objective Functionscited by 4
  8. 2018An Envelope for Davis-Yin Splitting and Strict Saddle Point Avoidancecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 2; 2026: 2.

7 papers.

  1. 2022Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimizationcited by 16
  2. 2024On Divergence Measures for Training GFlowNetscited by 6
  3. 2023Subsampling Error in Stochastic Gradient Langevin Diffusionscited by 1
  4. 2025A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularizationcited by 1
  5. 2026Weight-Decay Turns Transformer Loss Landscapes Villani: Functional-Analytic Foundations for Optimization and Generalizationcited by 1
  6. 2025Flatness-Aware Stochastic Gradient Langevin Dynamicscited by 0
  7. 2026Empirical Risk Minimization with f-Divergence Regularizationcited by 0