mlsanityData to 7 Oct 2026

An Improved Algorithm for Clustered Federated Learning

arXiv 2210.11538 ↗First posted 20 Oct 2022stat.ML cs.DC cs.ITcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we address the dichotomy between heterogeneous models and simultaneous training in Federated Learning (FL) via a clustering framework. We define a new clustering model for FL based on the (optimal) local models of the users: two users belong to the same cluster if their local models are close; otherwise they belong to different clusters. A standard algorithm for clustered FL is proposed in citeghoshefficient2021, called textttIFCA, which requires suitable initialization and the knowledge of hyper-parameters like the number of clusters (which is often quite difficult to obtain in practical applications) to converge. We propose an improved algorithm, Successive Refine Federated Clustering Algorithm (textttSR-FCA), which removes such restrictive assumptions. textttSR-FCA treats each user as a singleton cluster as an initialization, and then successively refine the cluster estimation via exploiting similar users belonging to the same cluster. In any intermediate step, textttSR-FCA uses a robust federated learning algorithm within each cluster to exploit simultaneous training and to correct clustering errors. Furthermore, textttSR-FCA does not require any good initialization (warm start), both in theory and practice. We show that with proper choice of learning rate, textttSR-FCA incurs arbitrarily small clustering error. Additionally, we validate the performance of our algorithm on standard FL datasets in non-convex problems like neural nets, and we show the benefits of textttSR-FCA over baselines.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

28 of 37 references.

References · 28

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2018LEAF: A Benchmark for Federated Settingscited by 780
  7. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  8. 2017Federated Multi-Task Learningcited by 669
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  10. 2016Federated Optimization: Distributed Machine Learning for On-Device Intelligencecited by 558
  11. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  12. 2020An Efficient Framework for Clustered Federated Learningcited by 505
16 more
  1. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  2. 2019Agnostic Federated Learningcited by 451
  3. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  4. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  5. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  6. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  7. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  8. 2017EMNIST: an extension of MNIST to handwritten letterscited by 212
  9. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  10. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 148
  11. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  12. 2021Multi-Center Federated Learning: Clients Clustering for Better Personalizationcited by 112
  13. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  14. 2016Statistical and Computational Guarantees of Lloyd's Algorithm and its Variantscited by 52
  15. 2021FedSoft: Soft Clustered Federated Learning with Proximal Local Updatingcited by 46
  16. 2018Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learningcited by 32

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2023Learning Personalized Models with Clustered System Identificationcited by 10
  2. 2024Towards Client Driven Federated Learningcited by 0