mlsanityData to 7 Oct 2026

Consistency and fluctuations for stochastic gradient Langevin dynamics

arXiv 1409.0578 ↗First posted 1 Sep 2014stat.MLcited by 65

Not on the map. Why?

Abstract (arXiv)

Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data sets is computationally expensive. Both the calculation of the acceptance probability and the creation of informed proposals usually require an iteration through the whole data set. The recently proposed stochastic gradient Langevin dynamics (SGLD) method circumvents this problem by generating proposals which are only based on a subset of the data, by skipping the accept-reject step and by using decreasing step-sizes sequence (δm)m ≥ 0. %Under appropriate Lyapunov conditions, We provide in this article a rigorous mathematical framework for analysing this algorithm. We prove that, under verifiable assumptions, the algorithm is consistent, satisfies a central limit theorem (CLT) and its asymptotic bias-variance decomposition can be characterized by an explicit functional of the step-sizes sequence (δm)m ≥ 0. We leverage this analysis to give practical recommendations for the notoriously difficult tuning of this algorithm: it is asymptotically optimal to use a step-size sequence of the type δm asymp m-1/3, leading to an algorithm whose mean squared error (MSE) decreases at rate O(m-1/3)

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Sep 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 9; 2022: 13; 2023: 7; 2024: 10; 2025: 17; 2026: 8.

65 papers.

Sort:

By area:

  1. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  2. 2021Differentiable Annealed Importance Sampling and the Perils of Gradient Noisecited by 36
  3. 2023Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlocited by 29
  4. 2021Time-independent Generalization Bounds for SGLD in Non-convex Settingscited by 27
  5. 2022Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constantscited by 24
  6. 2021On Convergence of Federated Averaging Langevin Dynamicscited by 14
  7. 2025Enhanced DACER Algorithm with High Diffusion Efficiencycited by 13
  8. 2021Deep Bayesian inference for seismic imaging with taskscited by 12
  9. 2022Scalable Bayesian Uncertainty Quantification for Neural Network Potentials: Promise and Pitfallscited by 11
  10. 2022Nonlinear Sufficient Dimension Reduction with a Stochastic Neural Networkcited by 10
  11. 2024Accelerating Approximate Thompson Sampling with Underdamped Langevin Monte Carlocited by 10
  12. 2022Interacting Contour Stochastic Gradient Langevin Dynamicscited by 9
38 more
  1. 2024Energy-based Hopfield Boosting for Out-of-Distribution Detectioncited by 9
  2. 2025The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditionscited by 9
  3. 2024Taming Score-Based Diffusion Priors for Infinite-Dimensional Nonlinear Inverse Problemscited by 7
  4. 2025Non-Reversible Langevin Algorithms for Constrained Samplingcited by 7
  5. 2021Stein ICP for Uncertainty Estimation in Point Cloud Matchingcited by 6
  6. 2022Energy-Based Models for Functional Data using Path Measure Tiltingcited by 5
  7. 2023Sampling and estimation on manifolds using the Langevin diffusioncited by 5
  8. 2025From Global to Local: A Scalable Benchmark for Local Posterior Samplingcited by 5
  9. 2021Decentralized Bayesian Learning with Metropolis-Adjusted Hamiltonian Monte Carlocited by 4
  10. 2021Geometry-informed irreversible perturbations for accelerated convergence of Langevin dynamicscited by 4
  11. 2022Tuning Stochastic Gradient Algorithms for Statistical Inference via Large-Sample Asymptoticscited by 4
  12. 2023Langevin Quasi-Monte Carlocited by 4
  13. 2025Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theorycited by 4
  14. 2025DIGing--SGLD: Decentralized and Scalable Langevin Sampling over Time--Varying Networkscited by 4
  15. 2022Bayesian Learning of Parameterised Quantum Circuitscited by 3
  16. 2022A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamicscited by 3
  17. 2024Causal-StoNet: Causal Inference for High-Dimensional Complex Datacited by 3
  18. 2024Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Samplingcited by 3
  19. 2026Stagewise Reinforcement Learning and the Geometry of the Regret Landscapecited by 3
  20. 2022A Dynamical System View of Langevin-Based Non-Convex Samplingcited by 2
  21. 2024Robust Approximate Sampling via Stochastic Gradient Barker Dynamicscited by 2
  22. 2024Generative vs. Discriminative modeling under the lens of uncertainty quantificationcited by 2
  23. 2025Parameter Expanded Stochastic Gradient Markov Chain Monte Carlocited by 2
  24. 2025Information-Theoretic Framework for Understanding Modern Machine-Learningcited by 2
  25. 2025Accelerating Constrained Sampling: A Large Deviations Approachcited by 2
  26. 2025Regime-Switching Langevin Monte Carlo Algorithmscited by 2
  27. 2025Anchored Langevin Algorithmscited by 2
  28. 2026Improved Guarantees for Langevin Monte Carlo with Average Smoothnesscited by 2
  29. 2026Deterministic Envelopes for Tamed SGLD: Decoupling Stochastic Gradient Noise and Localizing Tamingcited by 2
  30. 2021Quantifying the mini-batching error in Bayesian inference for Adaptive Langevin dynamicscited by 1
  31. 2022Functional Central Limit Theorem and Strong Law of Large Numbers for Stochastic Gradient Langevin Dynamicscited by 1
  32. 2023Bayes Complexity of Learners vs Overfittingcited by 1
  33. 2024Scalable and Calibrated Sampling for Bayesian Generalized Linear Mixed Model via Stochastic Gradient Markov Chain Monte Carlocited by 1
  34. 2024Drift to Remembercited by 1
  35. 2025JaxSGMC: Modular stochastic gradient MCMC in JAXcited by 1
  36. 2025Continuous Policy and Value Iteration for Relaxed Stochastic Control Problems and Its Convergencecited by 1
  37. 2025Bayesian neural networks with interpretable priors from Mercer kernelscited by 1
  38. 2026Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisationcited by 1

50 most cited of 65 All 65 (JSON)