Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective

arXiv 2405.20431 ↗First posted 30 May 2024cs.LG cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a promising paradigm that offers significant advancements in privacy-preserving, decentralized machine learning by enabling collaborative training of models across distributed devices without centralizing data. However, the practical deployment of FL systems faces a significant bottleneck: the communication overhead caused by frequently exchanging large model updates between numerous devices and a central server. This communication inefficiency can hinder training speed, model performance, and the overall feasibility of real-world FL applications. In this survey, we investigate various strategies and advancements made in communication-efficient FL, highlighting their impact and potential to overcome the communication challenges inherent in FL systems. Specifically, we define measures for communication efficiency, analyze sources of communication inefficiency in FL systems, and provide a taxonomy and comprehensive review of state-of-the-art communication-efficient FL methods. Additionally, we discuss promising future research directions for enhancing the communication efficiency of FL systems. By addressing the communication bottleneck, FL can be effectively applied and enable scalable and practical deployment across diverse applications that require privacy-preserving, decentralized machine learning, such as IoT, healthcare, or finance.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 83 of its 127 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

83 of 127 references.

References · 83

2015Distilling the Knowledge in a Neural Networkcited by 10,324
2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
2019Advances and Open Problems in Federated Learningcited by 3,046
2018Federated Optimization in Heterogeneous Networkscited by 2,987
2020Supervised Contrastive Learningcited by 2,829
2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
2020Adaptive Federated Optimizationcited by 1,046
2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
2021Model-Contrastive Federated Learningcited by 766

38 more

2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
2021Federated Learning Based on Dynamic Regularizationcited by 562
2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 383
2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 352
2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
2020FedML: A Research Library and Benchmark for Federated Machine Learningcited by 300
2018Dataset Distillationcited by 230
2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
2013Expectation Propagation for approximate Bayesian inferencecited by 174
2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 142
2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 141
2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 138
2016Distributed Mean Estimation with Limited Communicationcited by 136
2019One-Shot Federated Learningcited by 125
2020Optimal Client Sampling for Federated Learningcited by 124
2019Overcoming Forgetting in Federated Learning on Non-IID Datacited by 120
2021Decentralized Federated Averagingcited by 114
2020Distilled One-Shot Federated Learningcited by 109
2020UVeQFed: Universal Vector Quantization for Federated Learningcited by 107
2022FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extractioncited by 102
2021DENSE: Data-Free One-Shot Federated Learningcited by 100
2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
2021Preservation of the Global Knowledge by Not-True Distillation in Federated Learningcited by 87
2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 84
2022NVIDIA FLARE: Federated Learning from Simulation to Real-Worldcited by 80
2020Practical One-Shot Federated Learning for Cross-Silo Settingcited by 77
2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 69
2023FedExP: Speeding Up Federated Averaging via Extrapolationcited by 69
2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 63
2022Client Selection in Federated Learning: Principles, Challenges, and Opportunitiescited by 62

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.