Federated learning with hierarchical clustering of local updates to improve training on non-IID data
Not on the map. Why?
Abstract (arXiv)
Federated learning (FL) is a well established method for performing machine learning tasks over massively distributed data. However in settings where data is distributed in a non-iid (not independent and identically distributed) fashion -- as is typical in real world situations -- the joint model produced by FL suffers in terms of test set accuracy and/or communication costs compared to training on iid data. We show that learning a single joint model is often not optimal in the presence of certain types of non-iid data. In this work we present a modification to FL by introducing a hierarchical clustering step (FL+HC) to separate clusters of clients by the similarity of their local updates to the global joint model. Once separated, the clusters are trained independently and in parallel on specialised models. We present a robust empirical analysis of the hyperparameters for FL+HC for several iid and non-iid settings. We show how FL+HC allows model training to converge in fewer communication rounds (significantly so under some non-iid settings) compared to FL without clustering. Additionally, FL+HC allows for a greater percentage of clients to reach a target accuracy compared to standard FL. Finally we make suggestions for good default hyperparameters to promote superior performing specialised models without modifying the the underlying federated learning communication protocol.
Key facts. Cited by 222 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Apr 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 36; 2022: 43; 2023: 41; 2024: 57; 2025: 34; 2026: 11.
222 papers.
Sort:
By area:
- 2021Towards Personalized Federated Learningcited by 462
- 2021Federated Learning on Non-IID Data: A Surveycited by 345
- 2021A Field Guide to Federated Optimizationcited by 312
- 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2021Federated Learning for Open Bankingcited by 138
- 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
- 2021Federated Graph Classification over Non-IID Graphscited by 113
- 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
- 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
- 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
- 2021Flexible Clustered Federated Learning for Client-Level Data Distribution Shiftcited by 47
38 more
- 2021FedSoft: Soft Clustered Federated Learning with Proximal Local Updatingcited by 46
- 2022pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learningcited by 44
- 2021Communication Efficiency in Federated Learning: Achievements and Challengescited by 40
- 2023Efficient Personalized Federated Learning via Sparse Model-Adaptationcited by 38
- 2021Non-IID data and Continual Learning processes in Federated Learning: A long road aheadcited by 36
- 2021Emerging Trends in Federated Learning: From Model Fusion to Federated X Learningcited by 35
- 2021Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networkscited by 35
- 2022Federated Learning under Distributed Concept Driftcited by 35
- 2022FedPop: A Bayesian Approach for Personalised Federated Learningcited by 35
- 2022On the Convergence of Clustered Federated Learningcited by 31
- 2022Personalized Federated Learning With Graphcited by 29
- 2021Federated Mixture of Expertscited by 27
- 2023Privacy and Fairness in Federated Learning: on the Perspective of Trade-offcited by 26
- 2024FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Datacited by 25
- 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
- 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24
- 2021Personalized Federated Learning with Contextualized Generalizationcited by 23
- 2023Topology-aware Federated Learning in Edge Computing: A Comprehensive Surveycited by 23
- 2021Federated Learning for Intrusion Detection System: Concepts, Challenges and Future Directionscited by 22
- 2022FLIS: Clustered Federated Learning via Inference Similarity for Non-IID Data Distributioncited by 22
- 2022DYNAFED: Tackling Client Data Heterogeneity with Global Dynamicscited by 22
- 2023Bayesian Federated Learning: A Surveycited by 22
- 2023Federated Learning for Computer Visioncited by 22
- 2021Federated Learning with Taskonomy for Non-IID Datacited by 21
- 2022AsyncFedED: Asynchronous Federated Learning with Euclidean Distance based Adaptive Weight Aggregationcited by 19
- 2021A Decentralized Federated Learning Framework via Committee Mechanism with Convergence Guaranteecited by 16
- 2023Visual Prompt Based Personalized Federated Learningcited by 16
- 2023FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clusteringcited by 15
- 2025LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Datacited by 15
- 2023Exploiting Label Skews in Federated Learning with Model Concatenationcited by 14
- 2024Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Datacited by 14
- 2021Federated Learning for Short-term Residential Load Forecastingcited by 13
- 2024The Federation Strikes Back: A Survey of Federated Learning Privacy Attacks, Defenses, Applications, and Policy Landscapecited by 13
- 2025Federated Continual Learning: Concepts, Challenges, and Solutionscited by 13
- 2022Federated Learning and Meta Learning: Approaches, Applications, and Directionscited by 12
- 2023A Survey on Class Imbalance in Federated Learningcited by 12
- 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
- 2021Opportunistic Federated Learning: An Exploration of Egocentric Collaboration for Pervasive Computing Applicationscited by 11
50 most cited of 222 All 222 (JSON)
- 2026Upholding Robustness in Federated Learning: Trends, Emerging Strategies, and Research Opportunities23 Sep 2026
- 2026Efficiently Distributed Federated Learning17 Sep 2026
- 2026Beyond Non-IID: Learner--Client Distribution Mismatch in Federated Learning27 Aug 2026
- 2026Encryption-Compatible Clustered Federated Learning via Distributed Expectation-Maximization over Metadata30 Jul 2026
- 2026Hierarchical Federated Learning with Dynamic Clustering and Adaptive Regularization for Robust Infrastructure Inspection2 Jun 2026
- 2026Fed-BAC: Federated Bandit-Guided Additive Clustering in Hierarchical Federated Learning12 May 2026
- 2026FedGMI: Generative Model-Driven Federated Learning for Probabilistic Mixture Inference9 May 2026
- 2026Federated Hierarchical Clustering with Automatic Selection of Optimal Cluster Numbers13 Mar 2026
- 2026FedPrism: Adaptive Personalized Federated Learning under Non-IID Data9 Mar 2026
- 2026One-Shot Federated Clustering of Non-Independent Completely Distributed Data24 Jan 2026
- 2026FedBiCross: Personalized One-Shot Federated Learning on Medical Images5 Jan 2026
- 2025Clustered Federated Learning with Hierarchical Knowledge Distillation11 Dec 2025
38 more
- 2025SGFusion: Stochastic Geographic Gradient Fusion in Federated Learning27 Oct 2025
- 2025Personalized Collaborative Learning with Affinity-Based Variance Reduction17 Oct 2025
- 2025Beyond Aggregation: Guiding Clients in Heterogeneous Federated Learning27 Sep 2025
- 2025Towards Adapting Federated & Quantum Machine Learning for Network Intrusion Detection: A Survey24 Sep 2025
- 2025One-Shot Clustering for Federated Learning Under Clustering-Agnostic Assumption1 Sep 2025
- 2025Online Decentralized Federated Multi-task Learning With Trustworthiness in Cyber-Physical Systems31 Aug 2025
- 2025Enabling Trustworthy Federated Learning via Remote Attestation for Mitigating Byzantine Threats30 Aug 2025
- 2025GraphFedMIG: Tackling Class Imbalance in Federated Graph Learning via Mutual Information-Guided Generation14 Aug 2025
- 2025Long-Term Client Selection for Federated Learning with Non-IID Data: A Truthful Auction Approach7 Aug 2025
- 2025Federated Learning for Commercial Image Sources17 Jul 2025
- 2025FLsim: A Modular and Library-Agnostic Simulation Framework for Federated Learning15 Jul 2025
- 2025PPFL-RDSN: Privacy-Preserving Federated Learning-based Residual Dense Spatial Networks for Encrypted Lossy Image Reconstruction30 Jun 2025
- 2025FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learning26 Jun 2025
- 2025Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learning25 Jun 2025
- 2025FedBKD: Distilled Federated Learning to Embrace Gerneralization and Personalization on Non-IID Data25 Jun 2025
- 2025Topology-Aware Differential Privacy in Hierarchical Federated Learning24 Jun 2025
- 2025AdRo-FL: Informed and Secure Client Selection for Federated Learning in the Presence of Adversarial Aggregator21 Jun 2025
- 2025Clustered Federated Learning via Embedding Distributions9 Jun 2025
- 2025Towards Graph-Based Privacy-Preserving Federated Learning: ModelNet -- A ResNet-based Model Classification Dataset31 May 2025
- 2025Topology-Aware Knowledge Propagation in Decentralized Learning16 May 2025
- 2025Lazy But Effective: Collaborative Personalized Federated Learning with Heterogeneous Data5 May 2025
- 2025Stratify: Rethinking Federated Learning for Non-IID Data through Balanced Sampling18 Apr 2025
- 2025Query-based Knowledge Transfer for Heterogeneous Learning Environments12 Apr 2025
- 2025Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control7 Apr 2025
- 2025PA-CFL: Privacy-Adaptive Clustered Federated Learning for Transformer-Based Sales Forecasting on Heterogeneous Retail Data15 Mar 2025
- 2025One-Shot Clustering for Federated Learning6 Mar 2025
- 2025Federated Variational Inference for Bayesian Mixture Models18 Feb 2025
- 2025PLayer-FL: A Principled Approach to Personalized Layer-wise Cross-Silo Federated Learning12 Feb 2025
- 2025Federated Continual Learning: Concepts, Challenges, and Solutions10 Feb 2025
- 2025Interaction-Aware Gaussian Weighting for Clustered Federated Learning5 Feb 2025
- 2025A survey on Clustered Federated Learning: Taxonomy, Analysis and Applications29 Jan 2025
- 2025BrainGuard: Privacy-Preserving Multisubject Image Reconstructions from Brain Activities24 Jan 2025
- 2025LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Data3 Jan 2025
- 2024Federated Source-free Domain Adaptation for Classification: Weighted Cluster Aggregation for Unlabeled Data18 Dec 2024
- 2024Concurrent vertical and horizontal federated learning with fuzzy cognitive maps17 Dec 2024
- 2024Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixing17 Dec 2024
- 2024FedSub: Introducing Class-aware Subnetworks Fusion to Enhance Personalized Federated Learning13 Nov 2024
- 2024Fed-EC: Bandwidth-Efficient Clustering-Based Federated Learning For Autonomous Visual Robot Navigation6 Nov 2024
50 newest of 222 All 222 (JSON)
- 2021Federated Mixture of Expertscited by 27
- 2022Adaptive Expert Models for Personalization in Federated Learningcited by 4
- 2022Adaptive Expert Models for Personalization in Federated Learning15 Jun 2022
- 2021Federated Mixture of Experts14 Jul 2021
- 2021Towards Personalized Federated Learningcited by 462
- 2021Federated Learning on Non-IID Data: A Surveycited by 345
- 2021A Field Guide to Federated Optimizationcited by 312
- 2021FedBABU: Towards Enhanced Representation for Federated Image Classificationcited by 180
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
- 2021Federated Learning for Open Bankingcited by 138
- 2022Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challengescited by 137
- 2021Federated Graph Classification over Non-IID Graphscited by 113
- 2022Improving Generalization in Federated Learning by Seeking Flat Minimacited by 98
- 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
- 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
- 2021Flexible Clustered Federated Learning for Client-Level Data Distribution Shiftcited by 47
12 most cited of 219 All 219 (JSON)
- 2026Upholding Robustness in Federated Learning: Trends, Emerging Strategies, and Research Opportunities23 Sep 2026
- 2026Efficiently Distributed Federated Learning17 Sep 2026
- 2026Beyond Non-IID: Learner--Client Distribution Mismatch in Federated Learning27 Aug 2026
- 2026Encryption-Compatible Clustered Federated Learning via Distributed Expectation-Maximization over Metadata30 Jul 2026
- 2026Hierarchical Federated Learning with Dynamic Clustering and Adaptive Regularization for Robust Infrastructure Inspection2 Jun 2026
- 2026Fed-BAC: Federated Bandit-Guided Additive Clustering in Hierarchical Federated Learning12 May 2026
- 2026FedGMI: Generative Model-Driven Federated Learning for Probabilistic Mixture Inference9 May 2026
- 2026Federated Hierarchical Clustering with Automatic Selection of Optimal Cluster Numbers13 Mar 2026
- 2026FedPrism: Adaptive Personalized Federated Learning under Non-IID Data9 Mar 2026
- 2026One-Shot Federated Clustering of Non-Independent Completely Distributed Data24 Jan 2026
- 2026FedBiCross: Personalized One-Shot Federated Learning on Medical Images5 Jan 2026
- 2025Clustered Federated Learning with Hierarchical Knowledge Distillation11 Dec 2025
12 newest of 219 All 219 (JSON)