mlsanityData to 14 Oct 2026

Optimal Rate Adaption in Federated Learning with Compressed Communications

arXiv 2112.06694 ↗First posted 13 Dec 2021cs.LG cs.IT math.ITcited by 15

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) incurs high communication overhead, which can be greatly alleviated by compression for model updates. Yet the tradeoff between compression and model accuracy in the networked environment remains unclear and, for simplicity, most implementations adopt a fixed compression rate only. In this paper, we for the first time systematically examine this tradeoff, identifying the influence of the compression error on the final model accuracy with respect to the learning rate. Specifically, we factor the compression error of each global iteration into the convergence rate analysis under both strongly convex and non-convex loss functions. We then present an adaptation framework to maximize the final model accuracy by strategically adjusting the compression rate in each iteration. We have discussed the key implementation issues of our framework in practical networks with representative compression algorithms. Experiments over the popular MNIST and CIFAR-10 datasets confirm that our solution effectively reduces network traffic yet maintains high model accuracy in FL.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 20 of its 38 references that are in the map. First posted on arXiv 13 Dec 2021.

Builds on

20 of 38 references.

References · 20

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  3. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  4. 2020Federated Learning with Matched Averagingcited by 564
  5. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  6. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  7. 2019Federated Machine Learning: Concept and Applicationscited by 412
  8. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  9. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 408
  10. 2018Sparsified SGD with Memorycited by 376
  11. 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 352
  12. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
8 more
  1. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  2. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  3. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
  4. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  5. 2016Distributed Mean Estimation with Limited Communicationcited by 136
  6. 2019Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocationcited by 94
  7. 2016Gradient Descent Converges to Minimizerscited by 25
  8. 2021Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updatescited by 10

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 5; 2024: 4; 2025: 3.

15 papers.

Sort:

  1. 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
  2. 2022Federated Learning with Flexible Controlcited by 12
  3. 2023AnycostFL: Efficient On-Demand Federated Learning over Heterogeneous Edge Devicescited by 10
  4. 2023Incentive Mechanism Design for Unbiased Federated Learning with Randomized Client Participationcited by 8
  5. 2023Multi-Carrier NOMA-Empowered Wireless Federated Learning with Optimal Power and Bandwidth Allocationcited by 5
  6. 2023Data-Aware Gradient Compression for FL in Communication-Constrained Mobile Computingcited by 5
  7. 2023When MiniBatch SGD Meets SplitFed Learning:Convergence Analysis and Performance Evaluationcited by 4
  8. 2022Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learningcited by 3
  9. 2024Heterogeneity-Aware Cooperative Federated Edge Learning with Adaptive Computation and Communication Compressioncited by 2
  10. 2024A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learningcited by 1
  11. 2024ROSS: RObust decentralized Stochastic learning based on Shapley valuescited by 1
  12. 2025Communication-Efficient Device Scheduling for Federated Learning Using Lyapunov Optimizationcited by 1
3 more
  1. 2025γ-FedHT: Stepsize-Aware Hard-Threshold Gradient Compression in Federated Learningcited by 1
  2. 2025Taming Latency and Bandwidth: A Theoretical Framework and Adaptive Algorithm for Communication-Constrained Trainingcited by 1
  3. 2024FedGTST: Boosting Global Transferability of Federated Models via Statistics Tuningcited by 0