mlsanityData to 7 Oct 2026

Federated Learning: Strategies for Improving Communication Efficiency

arXiv 1610.05492 ↗First posted 18 Oct 2016cs.LGcited by 1,461

Not on the map. Why?

Abstract (arXiv)

Federated Learning is a machine learning setting where the goal is to train a high-quality centralized model while training data remains distributed over a large number of clients each with unreliable and relatively slow network connections. We consider learning algorithms for this setting where on each round, each client independently computes an update to the current model based on its local data, and communicates this update to a central server, where the client-side updates are aggregated to compute a new global model. The typical clients in this setting are mobile phones, and communication efficiency is of the utmost importance. In this paper, we propose two ways to reduce the uplink communication costs: structured updates, where we directly learn an update from a restricted space parametrized using a smaller number of variables, e.g. either low-rank or a random mask; and sketched updates, where we learn a full model update and then compress it using a combination of quantization, random rotations, and subsampling before sending it to the server. Experiments on both convolutional and recurrent networks show that the proposed methods can reduce the communication cost by two orders of magnitude.

Key facts. Cited by 1,461 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 343; 2022: 312; 2023: 266; 2024: 263; 2025: 186; 2026: 89.

1,461 papers.

Sort:

By area:

  1. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  2. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  3. 2021Federated Learning on Non-IID Data: A Surveycited by 345
  4. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 320
  5. 2021A Field Guide to Federated Optimizationcited by 312
  6. 2021FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Spacecited by 284
  7. 2021Training Neural Networks with Fixed Sparse Maskscited by 212
  8. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  9. 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
  10. 2021FedKD: Communication Efficient Federated Learning via Knowledge Distillationcited by 185
  11. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  12. 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 140
38 more
  1. 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
  2. 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
  3. 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
  4. 2022A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainabilitycited by 118
  5. 2021Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learningcited by 116
  6. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  7. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  8. 2021FedMix: Approximation of Mixup under Mean Augmented Federated Learningcited by 108
  9. 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
  10. 2021Distributionally Robust Federated Averagingcited by 104
  11. 2023Stochastic Controlled Averaging for Federated Learning with Communication Compressioncited by 100
  12. 2023Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmarkcited by 97
  13. 2023Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networkscited by 95
  14. 2023SLoRA: Federated Parameter Efficient Fine-Tuning of Language Modelscited by 92
  15. 2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
  16. 2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
  17. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  18. 2021Federated Learning: Opportunities and Challengescited by 83
  19. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 81
  20. 2021MARINA: Faster Non-Convex Distributed Learning with Compressioncited by 77
  21. 2021FedCM: Federated Learning with Client-level Momentumcited by 77
  22. 2022When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methodscited by 74
  23. 2021Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Datacited by 69
  24. 2021Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacycited by 66
  25. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
  26. 2021Eluding Secure Aggregation in Federated Learning via Model Inconsistencycited by 66
  27. 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
  28. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
  29. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 65
  30. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 64
  31. 2024AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networkscited by 64
  32. 2022FedVARP: Tackling the Variance Due to Partial Client Participation in Federated Learningcited by 62
  33. 2021Pufferfish: Communication-efficient Models At No Extra Costcited by 61
  34. 2021Provably Secure Federated Learning against Malicious Clientscited by 60
  35. 2024SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Modelscited by 60
  36. 2021Federated Multi-Armed Banditscited by 59
  37. 2021Privacy-Preserving Machine Learning: Methods, Challenges and Directionscited by 58
  38. 2021Personalized Federated Learning through Local Memorizationcited by 57

50 most cited of 1,461 All 1,461 (JSON)