Federated Learning Can Find Friends That Are Advantageous
Not on the map. Why?
Abstract (arXiv)
In Federated Learning (FL), the distributed nature and heterogeneity of client data present both opportunities and challenges. While collaboration among clients can significantly enhance the learning process, not all collaborations are beneficial; some may even be detrimental. In this study, we introduce a novel algorithm that assigns adaptive aggregation weights to clients participating in FL training, identifying those with data distributions most conducive to a specific learning objective. We demonstrate that our aggregation method converges no worse than the method that aggregates only the updates received from clients with the same data distribution. Furthermore, empirical evaluations consistently reveal that collaborations guided by our algorithm outperform traditional FL approaches. This underscores the critical role of judicious client selection and lays the foundation for more streamlined and effective FL implementations in the coming years.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 55 of its 79 references that are in the map. First posted on arXiv 7 Feb 2024.
Builds on
55 of 79 references.
References · 55
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 716
- 2018Local SGD Converges Fast and Communicates Littlecited by 595
- 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
- 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
- 2020An Efficient Framework for Clustered Federated Learningcited by 505
38 more
- 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
- 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
- 2018Sparsified SGD with Memorycited by 373
- 2021A Field Guide to Federated Optimizationcited by 312
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
- 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
- 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
- 2018Approximation Methods for Bilevel Programmingcited by 231
- 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
- 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
- 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
- 2013Optimal rates for zero-order convex optimization: the power of two function evaluationscited by 190
- 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
- 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 163
- 2020Better Theory for SGD in the Nonconvex Worldcited by 156
- 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
- 2020Survey of Personalization Techniques for Federated Learningcited by 144
- 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 136
- 2022ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!cited by 134
- 2020Optimal Client Sampling for Federated Learningcited by 123
- 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
- 2019Distributed Learning with Compressed Gradient Differencescited by 110
- 2015An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedbackcited by 105
- 2022A framework for bilevel optimization that enables stochastic and global variance reduction algorithmscited by 99
- 2019High probability generalization bounds for uniformly stable algorithms with nearly optimal ratecited by 97
- 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
- 2021A Single-Timescale Method for Stochastic Bilevel Optimizationcited by 69
- 2020Fast-Convergent Federated Learning with Adaptive Weightingcited by 67
- 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
- 2020Fast-Convergent Federated Learningcited by 61
- 2021A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inversecited by 53
- 2020A Better Alternative to Error Feedback for Communication-Efficient Distributed Learningcited by 42
- 2020Communication-Efficient Federated Learning via Optimal Client Samplingcited by 39
- 2022Will Bilevel Optimizers Benefit from Loopscited by 36
- 2022On the Convergence of Clustered Federated Learningcited by 31
- 2022Gradient-based Bi-level Optimization for Deep Learning: A Surveycited by 26
- 2022Communication-Efficient Topologies for Decentralized Learning with O(1) Consensus Ratecited by 25
- 2021Personalized Federated Learning with Contextualized Generalizationcited by 23
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2024: 1.
1 paper.