mlsanityData to 7 Oct 2026

On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo

arXiv 1802.05431 ↗First posted 15 Feb 2018stat.ML cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

We provide convergence guarantees in Wasserstein distance for a variety of variance-reduction methods: SAGA Langevin diffusion, SVRG Langevin diffusion and control-variate underdamped Langevin diffusion. We analyze these methods under a uniform set of assumptions on the log-posterior distribution, assuming it to be smooth, strongly convex and Hessian Lipschitz. This is achieved by a new proof technique combining ideas from finite-sum optimization and the analysis of sampling methods. Our sharp theoretical bounds allow us to identify regimes of interest where each method performs better than the others. Our theory is verified with experiments on real-world and synthetic datasets.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 2; 2024: 5; 2025: 2; 2026: 1.

19 papers.

Sort:

  1. 2022Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlocited by 63
  2. 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
  3. 2022Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimizationcited by 16
  4. 2022Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Conditioncited by 16
  5. 2021When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpointcited by 12
  6. 2024Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin Dynamicscited by 12
  7. 2023Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correctioncited by 5
  8. 2023ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compressioncited by 4
  9. 2022Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithmscited by 2
  10. 2025Random Reshuffling for Stochastic Gradient Langevin Dynamicscited by 2
  11. 2026Improved Guarantees for Langevin Monte Carlo with Average Smoothnesscited by 2
  12. 2021Constrained Ensemble Langevin Monte Carlocited by 1
7 more
  1. 2024On the SAGA algorithm with decreasing stepcited by 1
  2. 2024Log-concave Sampling from a Convex Body with a Barrier: a Robust and Unified Dikin Walkcited by 1
  3. 2025Quantum Speedups for Sampling and Non-convex Optimization with Stochastic Oraclescited by 1
  4. 2021A New Framework for Variance-Reduced Hamiltonian Monte Carlocited by 0
  5. 2022Preferential Subsampling for Stochastic Gradient Langevin Dynamicscited by 0
  6. 2024Continuous-time Riemannian SGD and SVRG Flows on Wasserstein Probabilistic Spacecited by 0
  7. 2024Scalable Monte Carlo for Bayesian Learningcited by 0