mlsanityData to 7 Oct 2026

Less than a Single Pass: Stochastically Controlled Stochastic Gradient Method

arXiv 1609.03261 ↗First posted 12 Sep 2016math.OC cs.DS cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

We develop and analyze a procedure for gradient-based optimization that we refer to as stochastically controlled stochastic gradient (SCSG). As a member of the SVRG family of algorithms, SCSG makes use of gradient estimates at two scales, with the number of updates at the faster scale being governed by a geometric random variable. Unlike most existing algorithms in this family, both the computation cost and the communication cost of SCSG do not necessarily scale linearly with the sample size n; indeed, these costs are independent of n when the target accuracy is low. An experimental evaluation on real datasets confirms the effectiveness of SCSG.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 3; 2023: 1; 2024: 2; 2025: 5.

17 papers.

Sort:

  1. 2021Fault-Tolerant Federated Reinforcement Learning with Theoretical Guaranteecited by 52
  2. 2025APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformerscited by 17
  3. 2021Faster Rates for Compressed Federated Learning with Client-Variance Reductioncited by 14
  4. 2021ANITA: An Optimal Loopless Accelerated Variance-Reduced Gradient Methodcited by 9
  5. 2025Provably Robust Federated Reinforcement Learningcited by 8
  6. 2023Adaptive, Doubly Optimal No-Regret Learning in Strongly Monotone and Exp-Concave Games with Gradient Feedbackcited by 6
  7. 2022Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimizationcited by 5
  8. 2022Sampling Through the Lens of Sequential Decision Makingcited by 4
  9. 2021A Variance Controlled Stochastic Method with Biased Estimation for Faster Non-convex Optimizationcited by 2
  10. 2024Probabilistic Guarantees of Stochastic Recursive Gradient in Non-Convex Finite Sum Problemscited by 2
  11. 2025Stopping Rules for Stochastic Gradient Descent via Anytime-Valid Confidence Sequencescited by 2
  12. 2021Decentralized Stochastic Proximal Gradient Descent with Variance Reduction over Time-varying Networkscited by 1
5 more
  1. 2022SADAM: Stochastic Adam, A Stochastic Operator for First-Order Gradient-based Optimizercited by 1
  2. 2021Escaping Saddle Points with Stochastically Controlled Stochastic Gradient Methodscited by 0
  3. 2024Stochastic Variance-Reduced Iterative Hard Thresholding in Graph Sparsity Optimizationcited by 0
  4. 2025Convergence Analysis of alpha-SVRG under Strong Convexitycited by 0
  5. 2025Permutation Randomization on Nonsmooth Nonconvex Optimization: A Theoretical and Experimental Studycited by 0