mlsanityData to 7 Oct 2026

Decentralized Federated Learning: Balancing Communication and Computing Costs

arXiv 2107.12048 ↗First posted 26 Jul 2021cs.LG cs.DCcited by 54

Not on the map. Why?

Abstract (arXiv)

Decentralized stochastic gradient descent (SGD) is a driving engine for decentralized federated learning (DFL). The performance of decentralized SGD is jointly influenced by inter-node communications and local updates. In this paper, we propose a general DFL framework, which implements both multiple local updates and multiple inter-node communications periodically, to strike a balance between communication efficiency and model consensus. It can provide a general decentralized SGD analytical framework. We establish strong convergence guarantees for the proposed DFL algorithm without the assumption of convex objectives. The convergence rate of DFL can be optimized to achieve the balance of communication and computing costs under constrained resources. For improving communication efficiency of DFL, compressed communication is further introduced to the proposed DFL as a new scheme, named DFL with compressed communication (C-DFL). The proposed C-DFL exhibits linear convergence for strongly convex objectives. Experiment results based on MNIST and CIFAR-10 datasets illustrate the superiority of DFL over traditional decentralized SGD methods and show that C-DFL further enhances communication efficiency.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 26 of its 48 references that are in the map. First posted on arXiv 26 Jul 2021.

Builds on

26 of 48 references.

References · 26

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  8. 2017Federated Multi-Task Learningcited by 669
  9. 2018Local SGD Converges Fast and Communicates Littlecited by 595
  10. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  11. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  12. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
14 more
  1. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
  2. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 218
  3. 2019On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimizationcited by 178
  4. 2013On the Convergence of Decentralized Gradient Descentcited by 158
  5. 2015Asynchronous Parallel Stochastic Gradient for Nonconvex Optimizationcited by 115
  6. 2018Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithmscited by 106
  7. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 75
  8. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
  9. 2019Decentralized Federated Learning: A Segmented Gossip Approachcited by 63
  10. 2017Collaborative Deep Learning in Fixed Topology Networkscited by 60
  11. 2017ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learningcited by 36
  12. 2019Robust Federated Learning with Noisy Communicationcited by 33
  13. 2016Decentralized Consensus Algorithm with Delayed and Stochastic Gradientscited by 7
  14. 2019Communication-Censored Linearized ADMM for Decentralized Consensus Optimizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 17; 2024: 11; 2025: 12; 2026: 7.

54 papers.

Sort:

By area:

  1. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
  2. 2024Decentralized Federated Learning: A Survey on Security and Privacycited by 25
  3. 2023Green Edge AI: A Contemporary Surveycited by 24
  4. 2023Topology-aware Federated Learning in Edge Computing: A Comprehensive Surveycited by 23
  5. 2024Unleashing the Power of Continual Learning on Non-Centralized Devices: A Surveycited by 23
  6. 2022An Energy and Carbon Footprint Analysis of Distributed and Federated Learningcited by 16
  7. 2023AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devicescited by 15
  8. 2023Fedstellar: A Platform for Decentralized Federated Learningcited by 14
  9. 2021Semi-Decentralized Federated Edge Learning with Data and Device Heterogeneitycited by 12
  10. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  11. 2022Confederated Learning: Federated Learning with Decentralized Edge Serverscited by 7
  12. 2023Advancements in Federated Learning: Models, Methods, and Privacycited by 7
38 more
  1. 2023Distributed Learning over Networks with Graph-Attention-Based Personalizationcited by 6
  2. 2023Stochastic Unrolled Federated Learningcited by 6
  3. 2023Blockchained Federated Learning for Internet of Things: A Comprehensive Surveycited by 5
  4. 2023Jamming Attacks on Decentralized Federated Learning in General Multi-Hop Wireless Networkscited by 4
  5. 2023On the Computation-Communication Trade-Off with A Flexible Gradient Tracking Approachcited by 4
  6. 2024Generalization Error Matters in Decentralized Learning Under Byzantine Attackscited by 4
  7. 2024NTK-DFL: Enhancing Decentralized Federated Learning in Heterogeneous Settings via Neural Tangent Kernelcited by 4
  8. 2023A Carbon Tracking Model for Federated Learning: Impact of Quantization and Sparsificationcited by 3
  9. 2024Distributed Event-Based Learning via ADMMcited by 3
  10. 2024Federated and Transfer Learning for Cancer Detection Based on Image Analysiscited by 3
  11. 2024FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Methodcited by 3
  12. 2022Communication Compression for Decentralized Learning with Operator Splitting Methodscited by 2
  13. 2023Sparse Decentralized Federated Learningcited by 2
  14. 2023Topology Learning for Heterogeneous Decentralized Federated Learning over Unreliable D2D Networkscited by 2
  15. 2024ProFe: Communication-Efficient Decentralized Federated Learning via Distillation and Prototypescited by 2
  16. 2022Privatized Graph Federated Learningcited by 1
  17. 2022Enhanced Decentralized Federated Learning based on Consensus in Connected Vehiclescited by 1
  18. 2023Collaborative Distributed Machine Learningcited by 1
  19. 2023Scheduling and Communication Schemes for Decentralized Federated Learningcited by 1
  20. 2024OCD-FL: A Novel Communication-Efficient Peer Selection-based Decentralized Federated Learningcited by 1
  21. 2025FLARE: Adaptive Multi-Dimensional Reputation for Robust Client Reliability in Federated Learningcited by 1
  22. 2025Time-varying Mixing Matrix Design for Energy-efficient Decentralized Federated Learningcited by 1
  23. 2026Decentralized Federated Learning With Energy Harvesting Devicescited by 1
  24. 2026A Secure and Private Distributed Bayesian Federated Learning Designcited by 1
  25. 2026Decentralized Federated Learning by Partial Message Exchangecited by 1
  26. 2023Decentralized federated learning methods for reducing communication cost and energy consumption in UAV networkscited by 0
  27. 2023FBChain: A Blockchain-based Federated Learning Model with Efficiency and Secure Communicationcited by 0
  28. 2024Decentralized EM Algorithm for Gaussian Mixtures under Data Heterogeneity and Partial Labelingcited by 0
  29. 2025S-VOTE: Similarity-based Voting for Client Selection in Decentralized Federated Learningcited by 0
  30. 2025CO-DEFEND: Continuous Decentralized Federated Learning for Secure DoH-Based Threat Detectioncited by 0
  31. 2025Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentumcited by 0
  32. 2025Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learningcited by 0
  33. 2025Chisme: Heterogeneity-Aware Gossip Learningcited by 0
  34. 2025Realistic Urban Traffic Generator using Decentralized Federated Learning for the SUMO simulatorcited by 0
  35. 2025A Hybrid Stochastic Gradient Tracking Method for Distributed Online Optimization Over Time-Varying Directed Networkscited by 0
  36. 2025Pareto-optimal Trade-offs Between Communication and Computation with Flexible Gradient Trackingcited by 0
  37. 2025Enhancing Communication Efficiency in FL with Adaptive Gradient Quantization and Communication Frequency Optimizationcited by 0
  38. 2025Adaptive Decentralized Federated Learning for Robust Optimizationcited by 0

50 most cited of 54 All 54 (JSON)