# FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering

arXiv: https://arxiv.org/abs/2407.07124 · HTML: https://mlsanity.com/paper/2407.07124/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Improving Federated Learning Personalization via Model Agnostic Meta Learning](https://mlsanity.com/paper/1909.12488/) (2019)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Federated Learning of a Mixture of Global and Local Models](https://mlsanity.com/paper/2002.05516/) (2020)
- [FedVision: An Online Visual Object Detection Platform Powered by Federated Learning](https://mlsanity.com/paper/2001.06202/) (2020)
- [Beyond Sharing Weights for Deep Domain Adaptation](https://mlsanity.com/paper/1603.06432/) (2016)
- [Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces](https://mlsanity.com/paper/2209.10526/) (2022)
- [FLIS: Clustered Federated Learning via Inference Similarity for Non-IID Data Distribution](https://mlsanity.com/paper/2208.09754/) (2022)
- [PGFed: Personalize Each Client's Global Objective for Federated Learning](https://mlsanity.com/paper/2212.01448/) (2022)
- [FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clustering](https://mlsanity.com/paper/2403.04144/) (2024)

## Built on by (most cited first, 9 of 9)

- [Towards Adapting Federated & Quantum Machine Learning for Network Intrusion Detection: A Survey](https://mlsanity.com/paper/2509.21389/) (2025)
- [Single-Round Clustered Federated Learning via Data Collaboration Analysis for Non-IID Data](https://mlsanity.com/paper/2601.09304/) (2026)
- [SEAFL: Enhancing Efficiency in Semi-Asynchronous Federated Learning through Adaptive Aggregation and Selective Training](https://mlsanity.com/paper/2503.05755/) (2025)
- [Forget to Generalize: Iterative Adaptation for Generalization in Federated Learning](https://mlsanity.com/paper/2602.04536/) (2026)
- [Heterogeneity-Aware Client Selection Methodology For Efficient Federated Learning](https://mlsanity.com/paper/2602.20450/) (2026)
- [FedPrism: Adaptive Personalized Federated Learning under Non-IID Data](https://mlsanity.com/paper/2603.08252/) (2026)
- [FedACT: Concurrent Federated Intelligence across Heterogeneous Data Sources](https://mlsanity.com/paper/2605.00011/) (2026)
- [FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity](https://mlsanity.com/paper/2608.01129/) (2026)
- [Opinion Dynamics-based Coalition Formation for Federated Learning in Heterogeneous IoT Systems](https://mlsanity.com/paper/2609.19695/) (2026)

## Abstract (arXiv)

Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative training of machine learning models over decentralized devices without exposing their local data. One of the major challenges in FL is the presence of uneven data distributions across client devices, violating the well-known assumption of independent-and-identically-distributed (IID) training samples in conventional machine learning. To address the performance degradation issue incurred by such data heterogeneity, clustered federated learning (CFL) shows its promise by grouping clients into separate learning clusters based on the similarity of their local data distributions. However, state-of-the-art CFL approaches require a large number of communication rounds to learn the distribution similarities during training until the formation of clusters is stabilized. Moreover, some of these algorithms heavily rely on a predefined number of clusters, thus limiting their flexibility and adaptability. In this paper, we propose em FedClust, a novel approach for CFL that leverages the correlation between local model weights and the data distribution of clients. em FedClust groups clients into clusters in a one-shot manner by measuring the similarity degrees among clients based on the strategically selected partial weights of locally trained models. We conduct extensive experiments on four benchmark datasets with different non-IID data settings. Experimental results demonstrate that em FedClust achieves higher model accuracy up to ~45% as well as faster convergence with a significantly reduced communication cost up to 2.7× compared to its state-of-the-art counterparts.