mlsanityData to 7 Oct 2026

LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression

arXiv 2403.04348 ↗First posted 7 Mar 2024math.OC cs.DC cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

In Distributed optimization and Learning, and even more in the modern framework of federated learning, communication, which is slow and costly, is critical. We introduce LoCoDL, a communication-efficient algorithm that leverages the two popular and effective techniques of Local training, which reduces the communication frequency, and Compression, in which short bitstreams are sent instead of full-dimensional vectors of floats. LoCoDL works with a large class of unbiased compressors that includes widely-used sparsification and quantization methods. LoCoDL provably benefits from local training and compression and enjoys a doubly-accelerated communication complexity, with respect to the condition number of the functions and the model dimension, in the general heterogenous regime with strongly convex functions. This is confirmed in practice, with LoCoDL outperforming existing algorithms.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

44 of 63 references.

References · 44

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  7. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  8. 2021A Field Guide to Federated Optimizationcited by 312
  9. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 192
  10. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
  11. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  12. 2020On Biased Compression for Distributed Learningcited by 140
32 more
  1. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  2. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
  3. 2019Distributed Learning with Compressed Gradient Differencescited by 110
  4. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 100
  5. 2020Acceleration for Compressed Gradient Descent in Distributed and Federated Optimizationcited by 90
  6. 2019Natural Compression for Distributed Deep Learningcited by 88
  7. 2019A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descentcited by 84
  8. 2020Variance-Reduced Methods for Machine Learningcited by 79
  9. 2021MARINA: Faster Non-Convex Distributed Learning with Compressioncited by 77
  10. 2019Stochastic Distributed Learning with Gradient Quantization and Variance Reductioncited by 64
  11. 2020Linearly Converging Error Compensated SGDcited by 61
  12. 2020Optimal Gradient Compression for Distributed and Federated Learningcited by 37
  13. 2022Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compressioncited by 33
  14. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  15. 2019A Double Residual Compression Algorithm for Efficient Distributed Learningcited by 31
  16. 2022EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compressioncited by 28
  17. 2022RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updatescited by 27
  18. 2023DoCoFL: Downlink Compression for Cross-Device Federated Learningcited by 27
  19. 2022Can 5th Generation Local Training Methods Support Client Sampling? Yes!cited by 22
  20. 2021Preserved central model for faster bidirectional compression in distributed settingscited by 21
  21. 2022Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Proxcited by 19
  22. 2022Federated Optimization Algorithms with Random Reshuffling and Gradient Compressioncited by 18
  23. 2022EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimizationcited by 17
  24. 2022GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexitycited by 17
  25. 2022DASHA: Distributed Nonconvex Optimization with Communication Compression, Optimal Oracle Complexity, and No Client Synchronizationcited by 15
  26. 2023Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?cited by 14
  27. 2021MURANA: A Generic Framework for Stochastic Variance-Reduced Optimizationcited by 13
  28. 2022Federated Random Reshuffling with Compression and Variance Reductioncited by 11
  29. 2022CompressedScaffnew: The First Theoretical Double Acceleration of Communication from Local Training and Compression in Distributed Optimizationcited by 11
  30. 2019One Method to Rule Them All: Variance Reduction for Data, Parameters and Many New Methodscited by 8
  31. 20232Direction: Theoretically Faster Distributed Training with Bidirectional Communication Compressioncited by 5
  32. 2023Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication Compressioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 6.

11 papers.

  1. 2025Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEscited by 4
  2. 2024Accelerated Methods with Compressed Communications for Distributed Optimization Problems under Data Similaritycited by 2
  3. 2024Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functionscited by 1
  4. 2025Decentralized Optimization with Topology-Independent Communicationcited by 1
  5. 2026BiCoLoR: Communication-Efficient Optimization with Bidirectional Compression and Local Trainingcited by 1
  6. 2026Gradient Compression May Hurt Generalization: A Remedy by Synthetic Data Guided Sharpness Aware Minimizationcited by 1
  7. 2026FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projectioncited by 1
  8. 2025Provable Reduction in Communication Rounds for Non-Smooth Convex Federated Learningcited by 0
  9. 2026SA-PEF: Step-Ahead Partial Error Feedback for Efficient Federated Learningcited by 0
  10. 2026LoDAdaC: a unified local training-based decentralized framework with adaptive gradients and compressed communicationcited by 0
  11. 2026LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averagingcited by 0