mlsanityData to 7 Oct 2026

Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning

arXiv 2408.14736 ↗First posted 27 Aug 2024cs.DC cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Current data compression methods, such as sparsification in Federated Averaging (FedAvg), effectively enhance the communication efficiency of Federated Learning (FL). However, these methods encounter challenges such as the straggler problem and diminished model performance due to heterogeneous bandwidth and non-IID (Independently and Identically Distributed) data. To address these issues, we introduce a bandwidth-aware compression framework for FL, aimed at improving communication efficiency while mitigating the problems associated with non-IID data. First, our strategy dynamically adjusts compression ratios according to bandwidth, enabling clients to upload their models at a close pace, thus exploiting the otherwise wasted time to transmit more data. Second, we identify the non-overlapped pattern of retained parameters after compression, which results in diminished client update signals due to uniformly averaged weights. Based on this finding, we propose a parameter mask to adjust the client-averaging coefficients at the parameter level, thereby more closely approximating the original updates, and improving the training convergence under heterogeneous environments. Our evaluations reveal that our method significantly boosts model accuracy, with a maximum improvement of 13% over the uncompressed FedAvg. Moreover, it achieves a 3.37× speedup in reaching the target accuracy compared to FedAvg with a Top-K compressor, demonstrating its effectiveness in accelerating convergence with compression. The integration of common compression techniques into our framework further establishes its potential as a versatile foundation for future cross-device, communication-efficient FL research, addressing critical challenges in FL and advancing the field of distributed machine learning.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 32 of its 55 references that are in the map. First posted on arXiv 27 Aug 2024.

Builds on

32 of 55 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  6. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  7. 2018LEAF: A Benchmark for Federated Settingscited by 780
  8. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  9. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  10. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  11. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  12. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
20 more
  1. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  2. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  3. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  4. 2021No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Datacited by 241
  5. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 219
  6. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  7. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  8. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  9. 2020Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approachcited by 80
  10. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
  11. 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
  12. 2019A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networkscited by 36
  13. 2022Faster Adaptive Federated Learningcited by 35
  14. 2023DisWOT: Student Architecture Search for Distillation WithOut Trainingcited by 29
  15. 2020Federated Recommendation System via Differential Privacycited by 26
  16. 2020Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusterscited by 23
  17. 2023EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantizationcited by 22
  18. 2020Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selectioncited by 20
  19. 2024FedImpro: Measuring and Improving Client Update in Federated Learningcited by 15
  20. 2024Towards Efficient Communication and Secure Federated Recommendation System via Low-rank Trainingcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
  2. 2025Layer-wise Update Aggregation with Recycling for Communication-Efficient Federated Learningcited by 1
  3. 2025From Privacy to Trust in the Agentic Era: A Taxonomy of Challenges in Trustworthy Federated Learning Through the Lens of Trust Report 2.0cited by 0
  4. 2026SplitLite: Low-Rank Residual Compression for Split Learningcited by 0