mlsanityData to 14 Oct 2026

Flexible Clustered Federated Learning for Client-Level Data Distribution Shift

arXiv 2108.09749 ↗First posted 22 Aug 2021cs.LG cs.DCcited by 47

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training data locally. Unlike the centralized training setting, the non-IID, imbalanced (statistical heterogeneity) and distribution shifted training data of FL is distributed in the federated network, which will increase the divergences between the local models and the global model, further degrading performance. In this paper, we propose a flexible clustered federated learning (CFL) framework named FlexCFL, in which we 1) group the training of clients based on the similarities between the clients' optimization directions for lower training divergence; 2) implement an efficient newcomer device cold start mechanism for framework scalability and practicality; 3) flexibly migrate clients to meet the challenge of client-level data distribution shift. FlexCFL can achieve improvements by dividing joint optimization into groups of sub-optimization and can strike a balance between accuracy and communication efficiency in the distribution shift environment. The convergence and complexity are analyzed to demonstrate the efficiency of FlexCFL. We also evaluate FlexCFL on several open datasets and made comparisons with related CFL frameworks. The results show that FlexCFL can significantly improve absolute test accuracy by +10.6% on FEMNIST compared to FedAvg, +3.5% on FashionMNIST compared to FedProx, +8.4% on MNIST compared to FeSEM. The experiment results show that FlexCFL is also communication efficient in the distribution shift environment.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Builds on 27 of its 47 references that are in the map. First posted on arXiv 22 Aug 2021.

Builds on

27 of 47 references.

References · 27

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  7. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  8. 2018Federated Learning with Non-IID Datacited by 1,108
  9. 2019Towards Federated Learning at Scale: System Designcited by 832
  10. 2017Federated Multi-Task Learningcited by 672
  11. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  12. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
15 more
  1. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  2. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  3. 2019Agnostic Federated Learningcited by 456
  4. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  5. 2019Federated Machine Learning: Concept and Applicationscited by 412
  6. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 409
  7. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  8. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 223
  9. 2019Client-Edge-Cloud Hierarchical Federated Learningcited by 202
  10. 2013Communication Efficient Distributed Optimization using an Approximate Newton-type Methodcited by 132
  11. 2020The Effect of Natural Distribution Shift on Question Answering Modelscited by 104
  12. 2019Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applicationscited by 64
  13. 2020TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecturecited by 12
  14. 2016On perfect clustering of high dimension, low sample size datacited by 10
  15. 2020Collaborative City Digital Twin For Covid-19 Pandemic: A Federated Learning Solutioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 12; 2024: 17; 2025: 10; 2026: 5.

47 papers.

Sort:

By area:

  1. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  2. 2023Advances and Challenges in Meta-Learning: A Technical Reviewcited by 65
  3. 2022Federated Learning under Distributed Concept Driftcited by 36
  4. 2023L-DAWA: Layer-wise Divergence Aware Weight Aggregation in Federated Self-Supervised Visual Representation Learningcited by 30
  5. 2024FedSelect: Personalized Federated Learning with Customized Selection of Parameters for Fine-Tuningcited by 27
  6. 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
  7. 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24
  8. 2023Exploiting Label Skews in Federated Learning with Model Concatenationcited by 14
  9. 2024Personalized Federated Learning via Feature Distribution Adaptationcited by 13
  10. 2022Auxo: Efficient Federated Learning via Scalable Client Clusteringcited by 8
  11. 2024Classifier Clustering and Feature Alignment for Federated Learning under Distributed Concept Driftcited by 8
  12. 2023A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Designcited by 5
35 more
  1. 2024Proximity-based Self-Federated Learningcited by 5
  2. 2023StoCFL: A Stochastically Clustered Federated Learning Framework for Non-IID Data with Dynamic Client Participationcited by 4
  3. 2024Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learningcited by 4
  4. 2023Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologiescited by 3
  5. 2024FedD2S: Personalized Data-Free Federated Knowledge Distillationcited by 3
  6. 2024FedBaF: Federated Learning Aggregation Biased by a Foundation Modelcited by 3
  7. 2024Federated Learning Clients Clustering with Adaptation to Data Driftscited by 3
  8. 2023Federated Two Stage Decoupling With Adaptive Personalization Layerscited by 2
  9. 2024Collaborative Visual Place Recognition through Federated Learningcited by 2
  10. 2024Distributionally Robust Clustered Federated Learning: A Case Study in Healthcarecited by 2
  11. 2025A survey on Clustered Federated Learning: Taxonomy, Analysis and Applicationscited by 2
  12. 2024FedTrans: Efficient Federated Learning via Multi-Model Transformationcited by 1
  13. 2024On-Demand Model and Client Deployment in Federated Learning with Deep Reinforcement Learningcited by 1
  14. 2024Federated Cross-Training Learners for Robust Generalization under Data Heterogeneitycited by 1
  15. 2025Privacy-Preserving Dataset Combinationcited by 1
  16. 2025Distilling A Universal Expert from Clustered Federated Learningcited by 1
  17. 2025CLoVE: Personalized Federated Learning through Clustering of Loss Vector Embeddingscited by 1
  18. 2023Failure-tolerant Distributed Learning for Anomaly Detection in Wireless Networkscited by 0
  19. 2023DAG-ACFL: Asynchronous Clustered Federated Learning based on DAG-DLTcited by 0
  20. 2023Personalized Federated Learning with Contextual Modulation and Meta-Learningcited by 0
  21. 2024Towards Client Driven Federated Learningcited by 0
  22. 2024Harnessing Increased Client Participation with Cohort-Parallel Federated Learningcited by 0
  23. 2024CDFL: Efficient Federated Human Activity Recognition using Contrastive Learning and Deep Clusteringcited by 0
  24. 2024Comparative Evaluation of Clustered Federated Learning Methodscited by 0
  25. 2025Extra Clients at No Extra Cost: Overcome Data Heterogeneity in Federated Learning with Filter Decompositioncited by 0
  26. 2025Lazy But Effective: Collaborative Personalized Federated Learning with Heterogeneous Datacited by 0
  27. 2025Addressing the Collaboration Dilemma in Low-Data Federated Learning via Transient Sparsitycited by 0
  28. 2025Clustered Federated Learning via Embedding Distributionscited by 0
  29. 2025Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learningcited by 0
  30. 2025Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learningcited by 0
  31. 2026Communication-Efficient Personalized Adaptation via Federated-Local Model Mergingcited by 0
  32. 2026FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environmentscited by 0
  33. 2026Personalized Federated Learning via Gaussian Generative Modelingcited by 0
  34. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0
  35. 2026Encryption-Compatible Clustered Federated Learning via Distributed Expectation-Maximization over Metadatacited by 0