mlsanityData to 14 Oct 2026

Data-Aware Gradient Compression for FL in Communication-Constrained Mobile Computing

arXiv 2311.07324 ↗First posted 13 Nov 2023cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) in mobile environments faces significant communication bottlenecks. Gradient compression has proven as an effective solution to this issue, offering substantial benefits in environments with limited bandwidth and metered data. Yet, it encounters severe performance drops in non-IID environments due to a one-size-fits-all compression approach, which does not account for the varying data volumes across workers. Assigning varying compression ratios to workers with distinct data distributions and volumes is therefore a promising solution. This work derives the convergence rate of distributed SGD with non-uniform compression, which reveals the intricate relationship between model convergence and the compression ratios applied to individual workers. Accordingly, we frame the relative compression ratio assignment as an n-variable chi-squared nonlinear optimization problem, constrained by a limited communication budget. We propose DAGC-R, which assigns conservative compression to workers handling larger data volumes. Recognizing the computational limitations of mobile devices, we propose the DAGC-A, which is computationally less demanding and enhances the robustness of compression in non-IID scenarios. Our experiments confirm that the DAGC-R and DAGC-A can speed up the training speed by up to 25.43% and 16.65% compared to the uniform compression respectively, when dealing with highly imbalanced data volume distribution and restricted communication.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 36 of its 57 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

36 of 57 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  8. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  9. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  10. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
  11. 2020Federated Learning with Matched Averagingcited by 564
  12. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
24 more
  1. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  2. 2020Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrievalcited by 309
  3. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 305
  4. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  5. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 238
  6. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  7. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
  8. 2022Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learningcited by 79
  9. 2018Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGDcited by 77
  10. 2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
  11. 2018Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimizationcited by 76
  12. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 54
  13. 2018Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communicationcited by 51
  14. 2021Rethinking gradient sparsification as total error minimizationcited by 41
  15. 2021An Efficient Statistical-based Gradient Compression Technique for Distributed Training Systemscited by 32
  16. 2021FEDZIP: A Compression Framework for Communication-Efficient Federated Learningcited by 27
  17. 2020Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGDcited by 26
  18. 2021Smoothness Matrices Beat Smoothness Constants: Better Communication Compression Techniques for Distributed Optimizationcited by 21
  19. 2020On Communication Compression for Distributed Optimization on Heterogeneous Datacited by 19
  20. 2021Optimal Rate Adaption in Federated Learning with Compressed Communicationscited by 15
  21. 2020Hierarchical Federated Learning through LAN-WAN Orchestrationcited by 13
  22. 2021Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updatescited by 10
  23. 2021Theoretically Better and Numerically Faster Distributed Optimization with Smoothness-Aware Quantization Techniquescited by 9
  24. 2022Learnings from Federated Learning in the Real worldcited by 6

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024Retraining-free Model Quantization via One-Shot Weight-Coupling Learningcited by 13
  2. 2025Optimizing Communication and Device Clustering for Clustered Federated Learning with Differential Privacycited by 4
  3. 2024Staleness-Centric Optimizations for Parallel Diffusion MoE Inferencecited by 1
  4. 2025Taming Latency and Bandwidth: A Theoretical Framework and Adaptive Algorithm for Communication-Constrained Trainingcited by 1
  5. 2025Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsitycited by 0