mlsanityData to 13 Oct 2026

Time-independent Generalization Bounds for SGLD in Non-convex Settings

arXiv 2111.12876 ↗First posted 25 Nov 2021stat.ML cs.LG math.OCcited by 27

Not on the map. Why?

Abstract (arXiv)

We establish generalization error bounds for stochastic gradient Langevin dynamics (SGLD) with constant learning rate under the assumptions of dissipativity and smoothness, a setting that has received increased attention in the sampling/optimization literature. Unlike existing bounds for SGLD in non-convex settings, ours are time-independent and decay to zero as the sample size increases. Using the framework of uniform stability, we establish time-independent bounds by exploiting the Wasserstein contraction property of the Langevin diffusion, which also allows us to circumvent the need to bound gradients using Lipschitz-like assumptions. Our analysis also supports variants of SGLD that use different discretization methods, incorporate Euclidean projections, or use non-isotropic noise.

Key facts. Cited by 27 papers in the map (data through 13 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 25 Nov 2021.

Builds on

21 of 33 references.

References · 21

  1. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 549
  2. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 302
  3. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  4. 2020Stability of Stochastic Gradient Descent on Nonsmooth Convex Lossescited by 172
  5. 2019Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimatescited by 119
  6. 2020Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descentcited by 118
  7. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  8. 2017Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimizationcited by 97
  9. 2019High probability generalization bounds for uniformly stable algorithms with nearly optimal ratecited by 97
  10. 2019Sharper bounds for uniformly stable algorithmscited by 92
  11. 2020Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithmscited by 89
  12. 2021Information-Theoretic Generalization Bounds for Stochastic Gradient Descentcited by 84
9 more
  1. 2017Generalization Bounds of SGLD for Non-convex Learning: Two Theoretical Viewpointscited by 82
  2. 2018Generalization Error Bounds for Noisy, Iterative Algorithmscited by 81
  3. 2014Consistency and fluctuations for stochastic gradient Langevin dynamicscited by 65
  4. 2019On Generalization Error Bounds of Noisy Gradient Methods for Non-Convex Learningcited by 57
  5. 2018The promises and pitfalls of Stochastic Gradient Langevin Dynamicscited by 56
  6. 2018Stability and Convergence Trade-off of Iterative Optimization Algorithmscited by 53
  7. 2019Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimizationcited by 38
  8. 2015Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Servercited by 17
  9. 2015Learning with Differential Privacy: Stability, Learnability and the Sufficiency and Necessity of ERM Principlecited by 16

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 7; 2023: 7; 2024: 5; 2025: 6; 2026: 2.

27 papers.

Sort:

By area:

  1. 2022Neural Networks Efficiently Learn Low-Dimensional Representations with SGDcited by 63
  2. 2023Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincar'e Inequalitycited by 31
  3. 2022Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GDcited by 19
  4. 2023Algorithmic Stability of Heavy-Tailed SGD with General Loss Functionscited by 15
  5. 2022Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squarescited by 12
  6. 2023Time-Independent Information-Theoretic Generalization Bounds for SGLDcited by 12
  7. 2022Generalisation under gradient descent via deterministic PAC-Bayescited by 9
  8. 2024Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Setscited by 9
  9. 2024Information-theoretic Generalization Analysis for Expected Calibration Errorcited by 9
  10. 2022Stability Based Generalization Bounds for Exponential Family Langevin Dynamicscited by 8
  11. 2023Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descentcited by 6
  12. 2025Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysiscited by 6
15 more
  1. 2023Select without Fear: Almost All Mini-Batch Schedules Generalize Optimallycited by 5
  2. 2022A sharp uniform-in-time error estimate for Stochastic Gradient Langevin Dynamicscited by 4
  3. 2024Generalization Error Matters in Decentralized Learning Under Byzantine Attackscited by 4
  4. 2025Algorithm- and Data-Dependent Generalization Bounds for Diffusion Modelscited by 4
  5. 2023Generalization Bounds for Label Noise Stochastic Gradient Descentcited by 3
  6. 2025Generalization Bounds for Markov Algorithms through Entropy Flow Computationscited by 2
  7. 2025Temperature is All You Need for Generalization in Langevin Dynamics and other Markov Processescited by 2
  8. 2025Beyond Real Data: Synthetic Data through the Lens of Regularizationcited by 2
  9. 2022Black-Box Generalization: Stability of Zeroth-Order Learningcited by 1
  10. 2023Geometric ergodicity of SGLD via reflection couplingcited by 1
  11. 2024Convex SGD: Generalization Without Early Stoppingcited by 1
  12. 2025Generalization of Gibbs and Langevin Monte Carlo Algorithms in the Interpolation Regimecited by 1
  13. 2024Provable Accuracy Bounds for Hybrid Dynamical Optimization and Samplingcited by 0
  14. 2026Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networkscited by 0
  15. 2026Generalization Bounds of Stochastic Gradient Descent in Homogeneous Neural Networkscited by 0