mlsanityData to 14 Oct 2026

FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models

arXiv 2403.09904 ↗First posted 14 Mar 2024cs.LG cs.AI cs.DCcited by 5

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server, while being respectful of privacy. A critical bottleneck in FL is the communication cost. A pivotal strategy to mitigate this burden is Local Training, which involves running multiple local stochastic gradient descent iterations between communication phases. Our work is inspired by the innovative Scaffnew algorithm, which has considerably advanced the reduction of communication complexity in FL. We introduce FedComLoc (Federated Compressed and Local Training), integrating practical and effective compression into Scaffnew to further enhance communication efficiency. Extensive experiments, using the popular TopK compressor and quantization, demonstrate its prowess in substantially reducing communication overheads in heterogeneous settings.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

34 of 47 references.

References · 34

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  4. 2018LEAF: A Benchmark for Federated Settingscited by 792
  5. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  6. 2019The State of Sparsity in Deep Neural Networkscited by 476
  7. 2021A Field Guide to Federated Optimizationcited by 315
  8. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  9. 2019Model Pruning Enables Efficient Federated Learning on Edge Devicescited by 222
  10. 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 213
  11. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 189
  12. 2019SCAFFOLD: Stochastic Controlled Averaging for Federated Learningcited by 175
22 more
  1. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
  2. 2019On the Convergence of Local Descent Methods in Federated Learningcited by 153
  3. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  4. 2021Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradientscited by 123
  5. 2020Local SGD: Unified Theory and New Efficient Methodscited by 88
  6. 2019First Analysis of Local GD on Heterogeneous Datacited by 81
  7. 2020From Local SGD to Local Fixed-Point Methods for Federated Learningcited by 74
  8. 2020Robust Quantization: One Model to Rule Them Allcited by 63
  9. 2021FedLab: A Flexible Federated Learning Frameworkcited by 55
  10. 2022NIPQ: Noise proxy-based Integrated Pseudo-Quantizationcited by 35
  11. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  12. 2021FedJAX: Federated learning simulation with JAXcited by 32
  13. 2022RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updatescited by 28
  14. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
  15. 2022GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexitycited by 18
  16. 2015SparkNet: Training Deep Networks in Sparkcited by 13
  17. 2022Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devicescited by 12
  18. 2022Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edgecited by 12
  19. 2018signSGD with Majority Vote is Communication Efficient And Fault Tolerantcited by 11
  20. 2022CompressedScaffnew: The First Theoretical Double Acceleration of Communication from Local Training and Compression in Distributed Optimizationcited by 11
  21. 2023Accurate Neural Network Pruning Requires Rethinking Sparse Optimizationcited by 10
  22. 2023Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2024Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimizationcited by 13
  2. 2024FedDM: Enhancing Communication Efficiency and Handling Data Heterogeneity in Federated Diffusion Modelscited by 5
  3. 2024Sparse-ProxSkip: Accelerated Sparse-to-Sparse Training in Federated Learningcited by 0
  4. 2025Provable Reduction in Communication Rounds for Non-Smooth Convex Federated Learningcited by 0
  5. 2026TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data
  2. 2024LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
  3. 2026A Full Compression Pipeline for Green Federated Learning in Communication-Constrained Environments
  4. 2021FEDZIP: A Compression Framework for Communication-Efficient Federated Learning
  5. 2021Communication-Efficient Federated Learning via Predictive Coding
  6. 2025FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learning
  7. 2022Sparse Random Networks for Communication-Efficient Federated Learning
  8. 2023FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications