mlsanityData to 7 Oct 2026

Heterogeneity for the Win: One-Shot Federated Clustering

arXiv 2103.00697 ↗First posted 1 Mar 2021cs.LGcited by 87

Not on the map. Why?

Abstract (arXiv)

In this work, we explore the unique challenges -- and opportunities -- of unsupervised federated learning (FL). We develop and analyze a one-shot federated clustering scheme, k-FED, based on the widely-used Lloyd's method for k-means clustering. In contrast to many supervised problems, we show that the issue of statistical heterogeneity in federated networks can in fact benefit our analysis. We analyse k-FED under a center separation assumption and compare it to the best known requirements of its centralized counterpart. Our analysis shows that in heterogeneous regimes where the number of clusters per device (k') is smaller than the total number of clusters over the network k, (k'≤ √k), we can use heterogeneity to our advantage -- significantly weakening the cluster separation requirements for k-FED. From a practical viewpoint, k-FED also has many desirable properties: it requires only round of communication, can run asynchronously, and can handle partial participation or node/network failures. We motivate our analysis with experiments on common FL benchmarks, and highlight the practical utility of one-shot clustering through use-cases in personalized FL and device sampling.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Builds on 15 of its 35 references that are in the map. First posted on arXiv 1 Mar 2021.

Builds on

15 of 35 references.

References · 15

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  5. 2018LEAF: A Benchmark for Federated Settingscited by 780
  6. 2017Federated Multi-Task Learningcited by 669
  7. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  8. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  9. 2019Agnostic Federated Learningcited by 451
  10. 2019Fair Resource Allocation in Federated Learningcited by 389
  11. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  12. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 196
3 more
  1. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 88
  2. 2013Distributed k-Means and k-Median Clustering on General Topologiescited by 24
  3. 2020Federated Matrix Factorization: Algorithm Design and Application to Data Clusteringcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 19; 2023: 16; 2024: 25; 2025: 14; 2026: 8.

87 papers.

Sort:

By area:

  1. 2021A Field Guide to Federated Optimizationcited by 312
  2. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 125
  3. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  4. 2022Orchestra: Unsupervised Federated Learning via Globally Consistent Clusteringcited by 45
  5. 2022On the Convergence of Clustered Federated Learningcited by 31
  6. 2022Towards Federated Clustering: A Federated Fuzzy c-Means Algorithm (FFCM)cited by 29
  7. 2022FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated Learningcited by 27
  8. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  9. 2023One-Shot Federated Conformal Predictioncited by 26
  10. 2023Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teachercited by 25
  11. 2022Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarkscited by 24
  12. 2023Federated Generative Learning with Foundation Modelscited by 23
38 more
  1. 2022DYNAFED: Tackling Client Data Heterogeneity with Global Dynamicscited by 22
  2. 2021Federated Learning with Taskonomy for Non-IID Datacited by 21
  3. 2022Machine Unlearning of Federated Clusterscited by 21
  4. 2021Exploiting Heterogeneity in Robust Federated Best-Arm Identificationcited by 20
  5. 2022Federated Learning from Only Unlabeled Data with Class-Conditional-Sharing Clientscited by 20
  6. 2024FedFisher: Leveraging Fisher Information for One-Shot Federated Learningcited by 15
  7. 2024Parametric Feature Transfer: One-shot Federated Learning with Foundation Modelscited by 13
  8. 2024Rethinking the Representation in Federated Unsupervised Learning with Non-IID Datacited by 12
  9. 2024Training Machine Learning models at the Edge: A Surveycited by 11
  10. 2024FuseFL: One-Shot Federated Learning through the Lens of Causality with Progressive Model Fusioncited by 11
  11. 2024Asynchronous Federated Clustering with Unknown Number of Clusterscited by 11
  12. 2022Secure Federated Clusteringcited by 10
  13. 2022A Snapshot of the Frontiers of Client Selection in Federated Learningcited by 9
  14. 2023Federated K-means Clusteringcited by 9
  15. 2022FLAME: Federated Learning Across Multi-device Environmentscited by 8
  16. 2022Auxo: Efficient Federated Learning via Scalable Client Clusteringcited by 8
  17. 2023Differentially Private Federated Clustering over Non-IID Datacited by 7
  18. 2025One-shot Federated Learning Methods: A Practical Guidecited by 6
  19. 2022Privacy-Preserving Federated Deep Clustering based on GANcited by 5
  20. 2023Federated Deep Multi-View Clustering with Global Self-Supervisioncited by 5
  21. 2023Contrastive encoder pre-training-based clustered federated learning for heterogeneous datacited by 5
  22. 2023Federated and distributed learning applications for electronic health records and structured medical data: A scoping reviewcited by 4
  23. 2024Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigationcited by 4
  24. 2022Combating Client Dropout in Federated Learning via Friend Model Substitutioncited by 3
  25. 2022A One-shot Framework for Distributed Clustered Learning in Heterogeneous Environmentscited by 3
  26. 2022Federated clustering with GAN-based data synthesiscited by 3
  27. 2024CCFC: Bridging Federated Clustering and Contrastive Learningcited by 3
  28. 2024CCFC++: Enhancing Federated Clustering through Feature Decorrelationcited by 3
  29. 2024Marginal and training-conditional guarantees in one-shot federated conformal predictioncited by 3
  30. 2024A Unified Solution to Diverse Heterogeneities in One-shot Federated Learningcited by 3
  31. 2026Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluationscited by 3
  32. 2023Federated K-Means Clustering via Dual Decomposition-based Distributed Optimizationcited by 2
  33. 2023Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsificationcited by 2
  34. 2023Federated learning-outcome prediction with multi-layer privacy protectioncited by 2
  35. 2024Asynchronous Multi-Server Federated Learning for Geo-Distributed Clientscited by 2
  36. 2024DFDG: Data-Free Dual-Generator Adversarial Distillation for One-Shot Federated Learningcited by 2
  37. 2025A survey on Clustered Federated Learning: Taxonomy, Analysis and Applicationscited by 2
  38. 2025Towards One-shot Federated Learning: Advances, Challenges, and Future Directionscited by 2

50 most cited of 87 All 87 (JSON)