mlsanityData to 7 Oct 2026

Federated Accelerated Stochastic Gradient Descent

arXiv 2006.08950 ↗First posted 16 Jun 2020cs.LG cs.DC math.OCcited by 85

Not on the map. Why?

Abstract (arXiv)

We propose Federated Accelerated Stochastic Gradient Descent (FedAc), a principled acceleration of Federated Averaging (FedAvg, also known as Local SGD) for distributed optimization. FedAc is the first provable acceleration of FedAvg that improves convergence speed and communication efficiency on various types of convex functions. For example, for strongly convex and smooth functions, when using M workers, the previous state-of-the-art FedAvg analysis can achieve a linear speedup in M if given M rounds of synchronization, whereas FedAc only requires M^frac13 rounds. Moreover, we prove stronger guarantees for FedAc when the objectives are third-order smooth. Our technique is based on a potential-based perturbed iterate analysis, a novel stability analysis of generalized accelerated SGD, and a strategic tradeoff between acceleration and stability.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 23; 2023: 15; 2024: 9; 2025: 6; 2026: 4.

85 papers.

Sort:

  1. 2021Federated Learning Based on Dynamic Regularizationcited by 555
  2. 2021A Field Guide to Federated Optimizationcited by 312
  3. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
  4. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  5. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  6. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  7. 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
  8. 2022Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environmentscited by 49
  9. 2021Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiencycited by 40
  10. 2021Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspectivecited by 38
  11. 2021Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devicescited by 36
  12. 2021The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communicationcited by 30
38 more
  1. 2021Private Federated Learning Without a Trusted Server: Optimal Algorithms for Convex Lossescited by 30
  2. 2022Acceleration of Federated Learning with Alleviated Forgetting in Local Trainingcited by 29
  3. 2023Understanding Generalization of Federated Learning via Stability: Heterogeneity Matterscited by 27
  4. 2022Private Non-Convex Federated Learning Without a Trusted Servercited by 26
  5. 2022Differentially Private Label Protection in Split Learningcited by 25
  6. 2021Federated Learning for Face Recognition with Gradient Correctioncited by 24
  7. 2021Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learningcited by 23
  8. 2021The Internet of Federated Things (IoFT): A Vision for the Future and In-depth Survey of Data-driven Approaches for Federated Learningcited by 23
  9. 2022Gradient Masked Averaging for Federated Learningcited by 22
  10. 2021GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learningcited by 21
  11. 2021Efficient Algorithms for Federated Saddle Point Optimizationcited by 20
  12. 2021Communication-efficient SGD: From Local SGD to One-Shot Averagingcited by 19
  13. 2021Vertical Federated Learning without Revealing Intersection Membershipcited by 19
  14. 2023Gradient Descent with Linearly Correlated Noise: Theory and Applications to Differential Privacycited by 18
  15. 2023Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learningcited by 16
  16. 2021On Bridging Generic and Personalized Federated Learning for Image Classificationcited by 15
  17. 2023RoBoSS: A Robust, Bounded, Sparse, and Smooth Loss Function for Supervised Learningcited by 15
  18. 2022Anchor Sampling for Federated Learning with Partial Client Participationcited by 14
  19. 2022Fast Composite Optimization and Statistical Recovery in Federated Learningcited by 12
  20. 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
  21. 2024The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communicationcited by 12
  22. 2022Over-the-Air Federated Learning via Second-Order Optimizationcited by 11
  23. 2021Local SGD Optimizes Overparameterized Neural Networks in Polynomial Timecited by 10
  24. 2021FedChain: Chained Algorithms for Near-Optimal Communication Cost in Federated Learningcited by 10
  25. 2022Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusioncited by 10
  26. 2021Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategiescited by 9
  27. 2022Federated Data Analytics: A Study on Linear Modelscited by 9
  28. 2021Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modelingcited by 8
  29. 2023Accelerating Fair Federated Learning: Adaptive Federated Adamcited by 8
  30. 2023Making Batch Normalization Great in Federated Deep Learningcited by 8
  31. 2023Federated Zeroth-Order Optimization using Trajectory-Informed Surrogate Gradientscited by 8
  32. 2024Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysiscited by 8
  33. 2022Confederated Learning: Federated Learning with Decentralized Edge Serverscited by 7
  34. 2022DPAUC: Differentially Private AUC Computation in Federated Learningcited by 7
  35. 2025Collaborative Bayesian Optimization via Wasserstein Barycenterscited by 7
  36. 2025Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Accelerationcited by 7
  37. 2021A Stochastic Newton Algorithm for Distributed Convex Optimizationcited by 6
  38. 2022Federated Learning on Adaptively Weighted Nodes by Bilevel Optimizationcited by 6

50 most cited of 85 All 85 (JSON)