mlsanityData to 7 Oct 2026

Communication-Efficient Federated Learning With Data and Client Heterogeneity

arXiv 2206.10032 ↗First posted 20 Jun 2022cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) enables large-scale distributed training of machine learning models, while still allowing individual nodes to maintain data locally. However, executing FL at scale comes with inherent practical challenges: 1) heterogeneity of the local node data distributions, 2) heterogeneity of node computational speeds (asynchrony), but also 3) constraints in the amount of communication between the clients and the server. In this work, we present the first variant of the classic federated averaging (FedAvg) algorithm which, at the same time, supports data heterogeneity, partial client asynchrony, and communication compression. Our algorithm comes with a novel, rigorous analysis showing that, in spite of these system relaxations, it can provide similar convergence to FedAvg in interesting parameter regimes. Experimental results in the rigorous LEAF benchmark on setups of up to 300 nodes show that our algorithm ensures fast convergence for standard federated tasks, improving upon prior quantized and asynchronous approaches.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 32 of its 61 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

32 of 61 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2019Advances and Open Problems in Federated Learningcited by 3,021
  6. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  7. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  8. 2020Adaptive Federated Optimizationcited by 1,036
  9. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
  10. 2019Towards Federated Learning at Scale: System Designcited by 826
  11. 2018LEAF: A Benchmark for Federated Settingscited by 780
  12. 2018Local SGD Converges Fast and Communicates Littlecited by 595
20 more
  1. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  2. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 334
  3. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 306
  4. 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 302
  5. 2021Federated Learning with Buffered Asynchronous Aggregationcited by 188
  6. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  7. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 150
  8. 2019Distributed Learning with Compressed Gradient Differencescited by 110
  9. 2019SAFA: a Semi-Asynchronous Protocol for Fast Federated Learning with Low Overheadcited by 82
  10. 2021MARINA: Faster Non-Convex Distributed Learning with Compressioncited by 77
  11. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 76
  12. 2022Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delayscited by 61
  13. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 60
  14. 2022Communication-Efficient Adaptive Federated Learningcited by 55
  15. 2020Bidirectional compression in heterogeneous settings for distributed or federated learning with partial participation: tight convergence guaranteescited by 45
  16. 2019Asynchronous Decentralized SGD with Quantized and Local Updatescited by 35
  17. 2020Moniqua: Modulo Quantized Communication in Decentralized SGDcited by 24
  18. 2021CANITA: Faster Rates for Distributed Convex Optimization with Communication Compressioncited by 23
  19. 2020Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Datacited by 15
  20. 2022DASHA: Distributed Nonconvex Optimization with Communication Compression, Optimal Oracle Complexity, and No Client Synchronizationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 3; 2026: 2.

9 papers.

  1. 2023AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithmscited by 16
  2. 2024Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimizationcited by 12
  3. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 12
  4. 2024Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailabilitycited by 9
  5. 2025Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimizationcited by 3
  6. 2025Quantum-Evolutionary Neural Networks for Multi-Agent Federated Learningcited by 0
  7. 2025Scalable Asynchronous Federated Modeling for Spatial Datacited by 0
  8. 2026Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Boundscited by 0
  9. 2026PubSwap: Public-Data Off-Policy Coordination for Federated RLVRcited by 0