mlsanityData to 7 Oct 2026

On the Convergence of Clustered Federated Learning

arXiv 2202.06187 ↗First posted 13 Feb 2022cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

Knowledge sharing and model personalization are essential components to tackle the non-IID challenge in federated learning (FL). Most existing FL methods focus on two extremes: 1) to learn a shared model to serve all clients with non-IID data, and 2) to learn personalized models for each client, namely personalized FL. There is a trade-off solution, namely clustered FL or cluster-wise personalized FL, which aims to cluster similar clients into one cluster, and then learn a shared model for all clients within a cluster. This paper is to revisit the research of clustered FL by formulating them into a bi-level optimization framework that could unify existing methods. We propose a new theoretical analysis framework to prove the convergence by considering the clusterability among clients. In addition, we embody this framework in an algorithm, named Weighted Clustered Federated Learning (WeCFL). Empirical analysis verifies the theoretical results and demonstrates the effectiveness of the proposed WeCFL under the proposed cluster-wise non-IID settings.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 13 Feb 2022.

Builds on

34 of 50 references.

References · 34

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  5. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  6. 2020Adaptive Federated Optimizationcited by 1,036
  7. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  8. 2021Model-Contrastive Federated Learningcited by 762
  9. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  10. 2020The Future of Digital Health with Federated Learningcited by 625
  11. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  12. 2018Local SGD Converges Fast and Communicates Littlecited by 595
22 more
  1. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  2. 2019Federated Learning with Personalization Layerscited by 533
  3. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  4. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  5. 2021Towards Personalized Federated Learningcited by 462
  6. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  7. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  8. 2021A Field Guide to Federated Optimizationcited by 312
  9. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  10. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  11. 2021Personalized Federated Learning using Hypernetworkscited by 210
  12. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 154
  13. 2020Federated Unsupervised Representation Learningcited by 92
  14. 2020FedGAN: Federated Generative Adversarial Networks for Distributed Datacited by 89
  15. 2021Heterogeneity for the Win: One-Shot Federated Clusteringcited by 87
  16. 2016Visualizing Large-scale and High-dimensional Datacited by 79
  17. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  18. 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
  19. 2019Real-World Image Datasets for Federated Learningcited by 30
  20. 2021Efficient and Less Centralized Federated Learningcited by 8
  21. 2021Federated Learning for Privacy-Preserving Open Innovation Future on Digital Healthcited by 8
  22. 2021Robust Convergence in Federated Learning through Label-wise Clusteringcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 13; 2024: 9; 2025: 6.

31 papers.

Sort:

  1. 2022Federated Learning from Pre-Trained Models: A Contrastive Learning Approachcited by 141
  2. 2022Federated Learning on Non-IID Graphs via Structural Knowledge Sharingcited by 88
  3. 2023Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Datacited by 40
  4. 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
  5. 2022Personalized Federated Learning With Graphcited by 29
  6. 2023Dual Personalization on Federated Recommendationcited by 19
  7. 2023Optimizing the Collaboration Structure in Cross-Silo Federated Learningcited by 19
  8. 2023Federated Recommendation with Additive Personalizationcited by 9
  9. 2023Differentially Private Federated Clustering over Non-IID Datacited by 7
  10. 2023FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimizationcited by 7
  11. 2024Hierarchical Federated Learning with Multi-Timescale Gradient Correctioncited by 7
  12. 2023Distributed Personalized Empirical Risk Minimizationcited by 6
19 more
  1. 2023Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clusteringcited by 5
  2. 2023StoCFL: A Stochastically Clustered Federated Learning Framework for Non-IID Data with Dynamic Client Participationcited by 4
  3. 2023Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologiescited by 3
  4. 2024A Bayesian Framework for Clustered Federated Learningcited by 3
  5. 2024Distributionally Robust Clustered Federated Learning: A Case Study in Healthcarecited by 2
  6. 2023Enhanced Federated Optimization: Adaptive Unbiased Client Sampling with Reduced Variancecited by 1
  7. 2024Federated Learning Can Find Friends That Are Advantageouscited by 1
  8. 2024Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimizationcited by 1
  9. 2025Distilling A Universal Expert from Clustered Federated Learningcited by 1
  10. 2025Clustered Federated Learning with Hierarchical Knowledge Distillationcited by 1
  11. 2023DAG-ACFL: Asynchronous Clustered Federated Learning based on DAG-DLTcited by 0
  12. 2024Towards Client Driven Federated Learningcited by 0
  13. 2024Personalized Additive Modeling for Multi-level Federated Learningcited by 0
  14. 2024Fast convergence of a Federated Expectation-Maximization Algorithmcited by 0
  15. 2024Federated One-Shot Ensemble Clusteringcited by 0
  16. 2025One-Shot Clustering for Federated Learningcited by 0
  17. 2025Communication-Efficient Module-Wise Federated Learning for Grasp Pose Detection in Cluttered Environmentscited by 0
  18. 2025One-Shot Clustering for Federated Learning Under Clustering-Agnostic Assumptioncited by 0
  19. 2025DPMM-CFL: Clustered Federated Learning via Dirichlet Process Mixture Model Nonparametric Clusteringcited by 0