Personalized Federated Learning with Multiple Known Clusters
Not on the map. Why?
Abstract (arXiv)
We consider the problem of personalized federated learning when there are known cluster structures within users. An intuitive approach would be to regularize the parameters so that users in the same cluster share similar model weights. The distances between the clusters can then be regularized to reflect the similarity between different clusters of users. We develop an algorithm that allows each cluster to communicate independently and derive the convergence results. We study a hierarchical linear model to theoretically demonstrate that our approach outperforms agents learning independently and agents learning a single shared weight. Finally, we demonstrate the advantages of our approach using both simulated and real-world data.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 67 references that are in the map. First posted on arXiv 28 Apr 2022.
Builds on
34 of 67 references.
References · 34
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
- 2017A Survey on Multi-Task Learningcited by 838
- 2019Towards Federated Learning at Scale: System Designcited by 826
- 2017Federated Multi-Task Learningcited by 669
- 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
- 2020Personalized Federated Learning with Moreau Envelopescited by 612
- 2018Local SGD Converges Fast and Communicates Littlecited by 595
- 2020An Efficient Framework for Clustered Federated Learningcited by 505
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
22 more
- 2020Adaptive Personalized Federated Learningcited by 353
- 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
- 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
- 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
- 2019SGD: General Analysis and Improved Ratescited by 206
- 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
- 2021Federated Multi-Task Learning under a Mixture of Distributionscited by 189
- 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
- 2019Hierarchical Federated Learning Across Heterogeneous Cellular Networkscited by 98
- 2019Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Recordscited by 70
- 2019Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loopcited by 68
- 2021Personalized Federated Learning: A Unified Framework and Universal Optimization Techniquescited by 34
- 2022Adaptive and Robust Multi-Task Learningcited by 33
- 2021FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learningcited by 19
- 2021Minimax Estimation for Personalized Federated Learning: An Alternative between FedAvg and Local Training?cited by 15
- 2019L-SVRG and L-Katyusha with Arbitrary Samplingcited by 14
- 2020Enhancing Privacy via Hierarchical Federated Learningcited by 10
- 2021ANITA: An Optimal Loopless Accelerated Variance-Reduced Gradient Methodcited by 9
- 2020Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problemscited by 8
- 2018Distributed Stochastic Multi-Task Learning with Graph Regularizationcited by 6
- 2021Error Compensated Loopless SVRG, Quartz, and SDCA for Distributed Optimizationcited by 4
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 1.
3 papers.