mlsanityData to 7 Oct 2026

ASAGA: Asynchronous Parallel SAGA

arXiv 1606.04809 ↗First posted 15 Jun 2016math.OC cs.LG stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

We describe ASAGA, an asynchronous parallel version of the incremental gradient algorithm SAGA that enjoys fast linear convergence rates. Through a novel perspective, we revisit and clarify a subtle but important technical issue present in a large fraction of the recent convergence rate proofs for asynchronous parallel optimization algorithms, and propose a simplification of the recently introduced "perturbed iterate" framework that resolves it. We thereby prove that ASAGA can obtain a theoretical linear speedup on multi-core systems even without sparsity assumptions. We present results of an implementation on a 40-core architecture illustrating the practical speedup as well as the hardware overhead.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 9; 2022: 2; 2023: 0; 2024: 1; 2025: 2; 2026: 3.

18 papers.

Sort:

By area:

  1. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  2. 2021Pufferfish: Communication-efficient Models At No Extra Costcited by 61
  3. 2021Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updatingcited by 37
  4. 2021Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devicescited by 36
  5. 2021Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guaranteescited by 17
  6. 2021Decoupled Greedy Learning of CNNs for Synchronous and Asynchronous Distributed Learningcited by 6
  7. 2020Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descentcited by 3
  8. 2021Sum-Rate-Distortion Function for Indirect Multiterminal Source Coding in Federated Learningcited by 2
  9. 2026Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneitycited by 2
  10. 2021Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossipingcited by 1
  11. 2022Distributed Dynamic Safe Screening Algorithms for Sparse Regularizationcited by 1
  12. 2022Scaling up Stochastic Gradient Descent for Non-convex Optimisationcited by 1
6 more
  1. 2024Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Frameworkcited by 1
  2. 2025A Class of Accelerated Fixed-Point-Based Methods with Delayed Inexact Oracles and Its Applicationscited by 1
  3. 2026Tight Analysis of Decentralized SGD: A Markov Chain Perspectivecited by 1
  4. 2021Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimizationcited by 0
  5. 2025Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Frameworkcited by 0
  6. 2026Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasetscited by 0