mlsanityData to 7 Oct 2026

Decentralized Federated Averaging

arXiv 2104.11375 ↗First posted 23 Apr 2021cs.DC stat.MLcited by 114

Not on the map. Why?

Abstract (arXiv)

Federated averaging (FedAvg) is a communication efficient algorithm for the distributed training with an enormous number of clients. In FedAvg, clients keep their data locally for privacy protection; a central parameter server is used to communicate between clients. This central server distributes the parameters to each client and collects the updated parameters from clients. FedAvg is mostly studied in centralized fashions, which requires massive communication between server and clients in each communication. Moreover, attacking the central server can break the whole system's privacy. In this paper, we study the decentralized FedAvg with momentum (DFedAvgM), which is implemented on clients that are connected by an undirected graph. In DFedAvgM, all clients perform stochastic gradient descent with momentum and communicate with their neighbors only. To further reduce the communication cost, we also consider the quantized DFedAvgM. We prove convergence of the (quantized) DFedAvgM under trivial assumptions; the convergence rate can be improved when the loss function satisfies the PL property. Finally, we numerically verify the efficacy of DFedAvgM.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Builds on 30 of its 61 references that are in the map. First posted on arXiv 23 Apr 2021.

Builds on

30 of 61 references.

References · 30

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2020Adaptive Federated Optimizationcited by 1,036
  7. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  8. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
  9. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  10. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  11. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  12. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 231
18 more
  1. 2017Asynchronous Decentralized Parallel Stochastic Gradient Descentcited by 159
  2. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  3. 2016Stochastic Variance Reduction for Nonconvex Optimizationcited by 143
  4. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 95
  5. 2017Recovery Guarantees for One-hidden-layer Neural Networkscited by 94
  6. 2018SGD and Hogwild! Convergence Without the Bounded Gradients Assumptioncited by 94
  7. 2019Peer-to-peer Federated Learning on Graphscited by 85
  8. 2019First Analysis of Local GD on Heterogeneous Datacited by 80
  9. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  10. 2019Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptionscited by 48
  11. 2016On Nonconvex Decentralized Gradient Descentcited by 42
  12. 2017Communication-Efficient Algorithms for Decentralized and Stochastic Optimizationcited by 42
  13. 2018Uniform Convergence of Gradients for Non-Convex Learning and Optimizationcited by 38
  14. 2019Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradientscited by 32
  15. 2020Decentralized Federated Learning via SGD over Wireless D2D Networkscited by 32
  16. 2019On Maintaining Linear Convergence of Distributed Learning and Optimization under Limited Communicationcited by 15
  17. 2018Distributed heavy-ball: A generalization and acceleration of first-order methods with gradient trackingcited by 14
  18. 2018Non-ergodic Convergence Analysis of Heavy-Ball Algorithmscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 11; 2023: 24; 2024: 35; 2025: 26; 2026: 16.

114 papers.

Sort:

By area:

  1. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
  2. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  3. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  4. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  5. 2022On the (In)security of Peer-to-Peer Decentralized Machine Learningcited by 23
  6. 2024Advances in APPFL: A Comprehensive and Extensible Federated Learning Frameworkcited by 19
  7. 2023Differentially Private Neural Tangent Kernels for Privacy-Preserving Data Generationcited by 15
  8. 2024Unlearning during Learning: An Efficient Federated Machine Unlearning Methodcited by 14
  9. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  10. 2025Decentralized Diffusion Modelscited by 12
  11. 2021Efficient and Reliable Overlay Networks for Decentralized Federated Learningcited by 9
  12. 2022Can Foundation Models Help Us Achieve Perfect Secrecy?cited by 9
38 more
  1. 2024Decentralized Directed Collaboration for Personalized Federated Learningcited by 9
  2. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  3. 2023Federated Learning with Manifold Regularization and Normalized Update Reaggregationcited by 8
  4. 2024One-Shot Sequential Federated Learning for Non-IID Data by Enhancing Local Model Diversitycited by 8
  5. 2021Clustered Federated Learning via Generalized Total Variation Minimizationcited by 7
  6. 2023The effect of network topologies on fully decentralized learning: a preliminary investigationcited by 7
  7. 2024Impact of network topology on the performance of Decentralized Federated Learningcited by 7
  8. 2024Federated Learning with MMD-based Early Stopping for Adaptive GNSS Interference Classificationcited by 7
  9. 2023Stochastic Unrolled Federated Learningcited by 6
  10. 2023DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learningcited by 6
  11. 2023Coordination-free Decentralised Federated Learning on Complex Networks: Overcoming Heterogeneitycited by 6
  12. 2023Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacycited by 5
  13. 2023Blockchain-Based Federated Learning: Incentivizing Data Sharing and Penalizing Dishonest Behaviorcited by 5
  14. 2023Asymmetrically Decentralized Federated Learningcited by 5
  15. 2024Robustness of Decentralised Learning to Nodes and Data Disruptioncited by 5
  16. 2022FedER: Federated Learning through Experience Replay and Privacy-Preserving Data Synthesiscited by 4
  17. 2022Using Anomaly Detection to Detect Poisoning Attacks in Federated Learning Applicationscited by 4
  18. 2023A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniquescited by 4
  19. 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
  20. 2024Decentralized Federated Learning with Model Caching on Mobile Agentscited by 4
  21. 2024Federated Prediction-Powered Inference from Decentralized Datacited by 4
  22. 2024NTK-DFL: Enhancing Decentralized Federated Learning in Heterogeneous Settings via Neural Tangent Kernelcited by 4
  23. 2025PFedDST: Personalized Federated Learning with Decentralized Selection Trainingcited by 4
  24. 2025From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challengescited by 4
  25. 2025FLoRIST: Singular Value Thresholding for Efficient and Accurate Federated Fine-Tuning of Large Language Modelscited by 4
  26. 2025DP-FedLoRA: Privacy-Enhanced Federated Fine-Tuning for On-Device Large Language Modelscited by 4
  27. 2023Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectivescited by 3
  28. 2023Rethinking Client Drift in Federated Learning: A Logit Perspectivecited by 3
  29. 2023SPIRT: A Fault-Tolerant and Reliable Peer-to-Peer Serverless ML Training Architecturecited by 3
  30. 2023Exploring the Impact of Disrupted Peer-to-Peer Communications on Fully Decentralized Learning in Disaster Scenarioscited by 3
  31. 2023Towards Understanding Generalization and Stability Gaps between Centralized and Decentralized Federated Learningcited by 3
  32. 2023Privacy Preservation in Artificial Intelligence and Extended Reality (AI-XR) Metaverses: A Surveycited by 3
  33. 2024Decentralized Personalized Federated Learning based on a Conditional Sparse-to-Sparser Schemecited by 3
  34. 2024Federated and Transfer Learning for Cancer Detection Based on Image Analysiscited by 3
  35. 2024On the Trade-off between Flatness and Optimization in Distributed Learningcited by 3
  36. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  37. 2022MULTI-FLGANs: Multi-Distributed Adversarial Networks for Non-IID distributioncited by 2
  38. 2022Effect of Homomorphic Encryption on the Performance of Training Federated Learning Generative Adversarial Networkscited by 2

50 most cited of 114 All 114 (JSON)