mlsanityData to 7 Oct 2026

A Hitting Time Analysis of Stochastic Gradient Langevin Dynamics

arXiv 1702.05575 ↗First posted 18 Feb 2017cs.LG math.OC stat.MLcited by 70

Not on the map. Why?

Abstract (arXiv)

We study the Stochastic Gradient Langevin Dynamics (SGLD) algorithm for non-convex optimization. The algorithm performs stochastic gradient descent, where in each step it injects appropriately scaled Gaussian noise to the update. We analyze the algorithm's hitting time to an arbitrary subset of the parameter space. Two results follow from our general theory: First, we prove that for empirical risk minimization, if the empirical risk is point-wise close to the (smooth) population risk, then the algorithm achieves an approximate local minimum of the population risk in polynomial time, escaping suboptimal local minima that only exist in the empirical risk. Second, we show that SGLD improves on one of the best known learnability results for learning linear classifiers under the zero-one loss.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 17; 2023: 15; 2024: 9; 2025: 9; 2026: 5.

70 papers.

Sort:

By area:

  1. 2021Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstructioncited by 77
  2. 2022High-dimensional limit theorems for SGD: Effective dynamics and critical scalingcited by 69
  3. 2023Revisiting the Noise Model of Stochastic Gradient Descentcited by 35
  4. 2024Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Datacited by 35
  5. 2022Testing distributional assumptions of learning algorithmscited by 30
  6. 2021Power-law escape rate of SGDcited by 29
  7. 2021Learning General Halfspaces with General Massart Noise under the Gaussian Distributioncited by 27
  8. 2022Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constantscited by 24
  9. 2021Accelerated replica exchange stochastic gradient Langevin diffusion enhanced Bayesian DeepONet for solving noisy parametric PDEscited by 22
  10. 2021Improved Algorithms for Efficient Active Learning Halfspaces with Massart and Tsybakov noisecited by 18
  11. 2021Forster Decomposition and Learning Halfspaces with Noisecited by 16
  12. 2021Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channelscited by 15
38 more
  1. 2021Provable Compressed Sensing with Generative Priors via Langevin Dynamicscited by 14
  2. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  3. 2022Optimal SQ Lower Bounds for Learning Halfspaces with Massart Noisecited by 14
  4. 2022Generalization Error Bounds for Deep Neural Networks Trained by SGDcited by 14
  5. 2023Time-Independent Information-Theoretic Generalization Bounds for SGLDcited by 12
  6. 2023Deep Insights into Noisy Pseudo Labeling on Graph Datacited by 11
  7. 2021Boosting in the Presence of Massart Noisecited by 10
  8. 2021ReLU Regression with Massart Noisecited by 10
  9. 2024Accelerating Approximate Thompson Sampling with Underdamped Langevin Monte Carlocited by 10
  10. 2022Interacting Contour Stochastic Gradient Langevin Dynamicscited by 9
  11. 2021Stochastic Gradient Langevin Dynamics with Variance Reductioncited by 7
  12. 2023Quantum Langevin Dynamics for Optimizationcited by 7
  13. 2024Understanding the Generalization Benefits of Late Learning Rate Decaycited by 6
  14. 2025A Near-optimal Algorithm for Learning Margin Halfspaces with Massart Noisecited by 6
  15. 2025Reheated Gradient-based Discrete Sampling for Combinatorial Optimizationcited by 6
  16. 2022Generalization Bounds for Gradient Methods via Discrete and Continuous Priorcited by 5
  17. 2023Efficient displacement convex optimization with particle gradient descentcited by 5
  18. 2023Near-Optimal Bounds for Learning Gaussian Halfspaces with Random Classification Noisecited by 5
  19. 2025Learning Noisy Halfspaces with a Margin: Massart is No Harder than Randomcited by 5
  20. 2025From Global to Local: A Scalable Benchmark for Local Posterior Samplingcited by 5
  21. 2021Decentralized Bayesian Learning with Metropolis-Adjusted Hamiltonian Monte Carlocited by 4
  22. 2023Stochastic Quantum Sampling for Non-Logconcave Distributions and Estimating Partition Functionscited by 4
  23. 2024Sampling from multi-modal distributions with polynomial query complexity in fixed dimension via reverse diffusioncited by 4
  24. 2025Testing Noise Assumptions of Learning Algorithmscited by 4
  25. 2022Convergence Error Analysis of Reflected Gradient Langevin Dynamics for Globally Optimizing Non-Convex Constrained Problemscited by 3
  26. 2022A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamicscited by 3
  27. 2023Constructing Semantics-Aware Adversarial Examples with a Probabilistic Perspectivecited by 3
  28. 2026On Forgetting and Stability of Score-based Generative modelscited by 3
  29. 2022Tackling benign nonconvexity with smoothing and stochastic gradientscited by 2
  30. 2022Weak Convergence of Approximate reflection coupling and its Application to Non-convex Optimizationcited by 2
  31. 2023Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimizationcited by 2
  32. 2025Smoothed Agnostic Learning of Halfspaces over the Hypercubecited by 2
  33. 2026On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvaturecited by 2
  34. 2022Continuous-time Analysis for Variational Inequalities: An Overview and Desideratacited by 1
  35. 2022On PAC Learning Halfspaces in Non-interactive Local Privacy Model with Public Unlabeled Datacited by 1
  36. 2022Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithmscited by 1
  37. 2022Recycling Scraps: Improving Private Learning by Leveraging Intermediate Checkpointscited by 1
  38. 2023Improved Langevin Monte Carlo for stochastic optimization via landscape modificationcited by 1

50 most cited of 70 All 70 (JSON)