# FLIS: Clustered Federated Learning via Inference Similarity for Non-IID Data Distribution

arXiv: https://arxiv.org/abs/2208.09754 · HTML: https://mlsanity.com/paper/2208.09754/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 20 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification](https://mlsanity.com/paper/2001.01526/) (2020)
- [Federated Learning of a Mixture of Global and Local Models](https://mlsanity.com/paper/2002.05516/) (2020)
- [Salvaging Federated Learning by Local Adaptation](https://mlsanity.com/paper/2002.04758/) (2020)
- [Distillation-Based Semi-Supervised Federated Learning for Communication-Efficient Collaborative Training with Non-IID Private Data](https://mlsanity.com/paper/2008.06180/) (2020)
- [Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning](https://mlsanity.com/paper/2006.12097/) (2020)
- [Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge Transfer](https://mlsanity.com/paper/1912.11279/) (2019)
- [FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion](https://mlsanity.com/paper/2110.11027/) (2021)
- [Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity](https://mlsanity.com/paper/2105.00562/) (2021)

## Built on by (most cited first, 22 of 22)

- [Towards Building the Federated GPT: Federated Instruction Tuning](https://mlsanity.com/paper/2305.05644/) (2023)
- [Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces](https://mlsanity.com/paper/2209.10526/) (2022)
- [Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks](https://mlsanity.com/paper/2209.15595/) (2022)
- [Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents](https://mlsanity.com/paper/2312.01537/) (2023)
- [Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions](https://mlsanity.com/paper/2411.12377/) (2024)
- [Federated Continual Learning: Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2502.07059/) (2025)
- [When Do Curricula Work in Federated Learning?](https://mlsanity.com/paper/2212.12712/) (2022)
- [FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering](https://mlsanity.com/paper/2407.07124/) (2024)
- [Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learning](https://mlsanity.com/paper/2405.15632/) (2024)
- [A Bayesian Framework for Clustered Federated Learning](https://mlsanity.com/paper/2410.15473/) (2024)
- [Federated Two Stage Decoupling With Adaptive Personalization Layers](https://mlsanity.com/paper/2308.15821/) (2023)
- [Distributionally Robust Clustered Federated Learning: A Case Study in Healthcare](https://mlsanity.com/paper/2410.07039/) (2024)
- [A survey on Clustered Federated Learning: Taxonomy, Analysis and Applications](https://mlsanity.com/paper/2501.17512/) (2025)
- [TPFL: Tsetlin-Personalized Federated Learning with Confidence-Based Clustering](https://mlsanity.com/paper/2409.10392/) (2024)
- [rAge-k: Communication-Efficient Federated Learning Using Age Factor](https://mlsanity.com/paper/2410.22192/) (2024)
- [Energy-Efficient Federated Learning for AIoT using Clustering Methods](https://mlsanity.com/paper/2505.09704/) (2025)
- [FedSiKD: Clients Similarity and Knowledge Distillation: Addressing Non-i.i.d. and Constraints in Federated Learning](https://mlsanity.com/paper/2402.09095/) (2024)
- [SGFusion: Stochastic Geographic Gradient Fusion in Federated Learning](https://mlsanity.com/paper/2510.23455/) (2025)
- [ILoRA: Federated Learning with Low-Rank Adaptation for Heterogeneous Client Aggregation](https://mlsanity.com/paper/2511.16069/) (2025)
- [FedBiCross: Personalized One-Shot Federated Learning on Medical Images](https://mlsanity.com/paper/2601.01901/) (2026)
- [Enhancing Cross-Problem Vehicle Routing via Federated Learning](https://mlsanity.com/paper/2604.10652/) (2026)
- [From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning](https://mlsanity.com/paper/2606.10595/) (2026)

## Abstract (arXiv)

Classical federated learning approaches yield significant performance degradation in the presence of Non-IID data distributions of participants. When the distribution of each local dataset is highly different from the global one, the local objective of each client will be inconsistent with the global optima which incur a drift in the local updates. This phenomenon highly impacts the performance of clients. This is while the primary incentive for clients to participate in federated learning is to obtain better personalized models. To address the above-mentioned issue, we present a new algorithm, FLIS, which groups the clients population in clusters with jointly trainable data distributions by leveraging the inference similarity of clients' models. This framework captures settings where different groups of users have their own objectives (learning tasks) but by aggregating their data with others in the same cluster (same learning task) to perform more efficient and personalized federated learning. We present experimental results to demonstrate the benefits of FLIS over the state-of-the-art benchmarks on CIFAR-100/10, SVHN, and FMNIST datasets. Our code is available at https://github.com/MMorafah/FLIS.