mlsanityData to 7 Oct 2026

Asynchronous stochastic convex optimization

arXiv 1508.00882 ↗First posted 4 Aug 2015math.OC stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

We show that asymptotically, completely asynchronous stochastic gradient procedures achieve optimal (even to constant factors) convergence rates for the solution of convex optimization problems under nearly the same conditions required for asymptotic optimality of standard stochastic gradient procedures. Roughly, the noise inherent to the stochastic approximation scheme dominates any noise from asynchrony. We also give empirical evidence demonstrating the strong performance of asynchronous, parallel stochastic optimization schemes, demonstrating that the robustness inherent to stochastic approximation problems allows substantially faster parallel and asynchronous solution methods.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Aug 2015.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 10; 2022: 2; 2023: 2; 2024: 1; 2025: 1.

17 papers.

Sort:

By area:

  1. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  2. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 76
  3. 20211-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speedcited by 74
  4. 2021Asynchronous Stochastic Optimization Robust to Arbitrary Delayscited by 32
  5. 20211-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speedcited by 24
  6. 2021Distributed Learning Systems with First-order Methodscited by 17
  7. 2021Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guaranteescited by 17
  8. 2021Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updatescited by 16
  9. 2021Towards Demystifying Serverless Machine Learning Trainingcited by 16
  10. 2021Accelerated, Optimal, and Parallel: Some Results on Model-Based Stochastic Optimizationcited by 14
  11. 2023Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learningcited by 10
  12. 2023The Sample Complexity Of ERMs In Stochastic Convex Optimizationcited by 8
5 more
  1. 2022Clustered Federated Learning based on Nonconvex Pairwise Fusioncited by 4
  2. 2020Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descentcited by 3
  3. 2021Distributed stochastic optimization with large delayscited by 3
  4. 2024Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batchingcited by 1
  5. 2025Sharp Gaussian approximations for Decentralized Federated Learningcited by 1