mlsanityData to 7 Oct 2026

HADFL: Heterogeneity-aware Decentralized Federated Learning Framework

arXiv 2111.08274 ↗First posted 16 Nov 2021cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated learning (FL) supports training models on geographically distributed devices. However, traditional FL systems adopt a centralized synchronous strategy, putting high communication pressure and model generalization challenge. Existing optimizations on FL either fail to speedup training on heterogeneous devices or suffer from poor communication efficiency. In this paper, we propose HADFL, a framework that supports decentralized asynchronous training on heterogeneous devices. The devices train model locally with heterogeneity-aware local steps using local data. In each aggregation cycle, they are selected based on probability to perform model synchronization and aggregation. Compared with the traditional FL system, HADFL can relieve the central server's communication pressure, efficiently utilize heterogeneous computing power, and can achieve a maximum speedup of 3.15x than decentralized-FedAvg and 4.68x than Pytorch distributed training scheme, respectively, with almost no loss of convergence accuracy.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 13 of its 30 references that are in the map. First posted on arXiv 16 Nov 2021.

Builds on

13 of 30 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2019Towards Federated Learning at Scale: System Designcited by 826
  5. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
  6. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  7. 2018Horovod: fast and easy distributed deep learning in TensorFlowcited by 275
  8. 2019Asynchronous Federated Optimizationcited by 264
  9. 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
  10. 2018A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillationcited by 110
  11. 2019Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregationcited by 99
  12. 2019SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overheadcited by 82
1 more
  1. 2019Decentralized Federated Learning: A Segmented Gossip Approachcited by 63

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 2.

5 papers.

  1. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
  2. 2022GitFL: Adaptive Asynchronous Federated Learning using Version Controlcited by 10
  3. 2023Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learningcited by 5
  4. 2024When Foresight Pruning Meets Zeroth-Order Optimization: Efficient Federated Learning for Low-Memory Devicescited by 2
  5. 2024Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixingcited by 1