# Multi-Center Federated Learning: Clients Clustering for Better Personalization

arXiv: https://arxiv.org/abs/2108.08647 · HTML: https://mlsanity.com/paper/2108.08647/

## Key facts

- Cited by 113 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 87 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 30 of 45)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Federated Learning with Matched Averaging](https://mlsanity.com/paper/2002.06440/) (2020)
- [Federated Learning with Personalization Layers](https://mlsanity.com/paper/1912.00818/) (2019)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Differentially Private Federated Learning: A Client Level Perspective](https://mlsanity.com/paper/1712.07557/) (2017)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection](https://mlsanity.com/paper/1907.09693/) (2019)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data](https://mlsanity.com/paper/1811.11479/) (2018)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Federated Learning of a Mixture of Global and Local Models](https://mlsanity.com/paper/2002.05516/) (2020)

## Built on by (most cited first, 30 of 113)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Federated Learning from Pre-Trained Models: A Contrastive Learning Approach](https://mlsanity.com/paper/2209.10083/) (2022)
- [Federated Learning for Open Banking](https://mlsanity.com/paper/2108.10749/) (2021)
- [Federated Learning on Non-IID Graphs via Structural Knowledge Sharing](https://mlsanity.com/paper/2211.13009/) (2022)
- [Decentralized Federated Learning: A Survey and Perspective](https://mlsanity.com/paper/2306.01603/) (2023)
- [FedSoft: Soft Clustered Federated Learning with Proximal Local Updating](https://mlsanity.com/paper/2112.06053/) (2021)
- [Prompt Federated Learning for Weather Forecasting: Toward Foundation Models on Meteorological Data](https://mlsanity.com/paper/2301.09152/) (2023)
- [Personalized Federated Learning With Graph](https://mlsanity.com/paper/2203.00829/) (2022)
- [FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Data](https://mlsanity.com/paper/2403.16460/) (2024)
- [A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacy](https://mlsanity.com/paper/2302.10637/) (2023)
- [Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey](https://mlsanity.com/paper/2412.13840/) (2024)
- [MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligence](https://mlsanity.com/paper/2304.01950/) (2023)
- [Dual Personalization on Federated Recommendation](https://mlsanity.com/paper/2301.08143/) (2023)
- [Optimizing the Collaboration Structure in Cross-Silo Federated Learning](https://mlsanity.com/paper/2306.06508/) (2023)
- [FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering](https://mlsanity.com/paper/2301.12379/) (2023)
- [LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Data](https://mlsanity.com/paper/2501.01850/) (2025)
- [Exploiting Label Skews in Federated Learning with Model Concatenation](https://mlsanity.com/paper/2312.06290/) (2023)
- [Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data](https://mlsanity.com/paper/2206.00686/) (2022)
- [An Optimal Transport Approach to Personalized Federated Learning](https://mlsanity.com/paper/2206.02468/) (2022)
- [Federated Learning and Meta Learning: Approaches, Applications, and Directions](https://mlsanity.com/paper/2210.13111/) (2022)
- [Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies](https://mlsanity.com/paper/2109.03999/) (2021)
- [Federated Recommendation with Additive Personalization](https://mlsanity.com/paper/2301.09109/) (2023)
- [Co-clustering for Federated Recommender System](https://mlsanity.com/paper/2411.01690/) (2024)
- [Federated Learning for Privacy-Preserving Open Innovation Future on Digital Health](https://mlsanity.com/paper/2108.10761/) (2021)
- [Auxo: Efficient Federated Learning via Scalable Client Clustering](https://mlsanity.com/paper/2210.16656/) (2022)
- [A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications](https://mlsanity.com/paper/2403.01387/) (2024)
- [FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization](https://mlsanity.com/paper/2305.02894/) (2023)
- [FedGKD: Unleashing the Power of Collaboration in Federated Graph Neural Networks](https://mlsanity.com/paper/2309.09517/) (2023)
- [Dynamic Heterogeneous Federated Learning with Multi-Level Prototypes](https://mlsanity.com/paper/2312.09881/) (2023)
- [Influence-Oriented Personalized Federated Learning](https://mlsanity.com/paper/2410.03315/) (2024)

## Abstract (arXiv)

Personalized decision-making can be implemented in a Federated learning (FL) framework that can collaboratively train a decision model by extracting knowledge across intelligent clients, e.g. smartphones or enterprises. FL can mitigate the data privacy risk of collaborative training since it merely collects local gradients from users without access to their data. However, FL is fragile in the presence of statistical heterogeneity that is commonly encountered in personalized decision-making, e.g., non-IID data over different clients. Existing FL approaches usually update a single global model to capture the shared knowledge of all users by aggregating their gradients, regardless of the discrepancy between their data distributions. By comparison, a mixture of multiple global models could capture the heterogeneity across various clients if assigning the client to different global models (i.e., centers) in FL. To this end, we propose a novel multi-center aggregation mechanism to cluster clients using their models' parameters. It learns multiple global models from data as the cluster centers, and simultaneously derives the optimal matching between users and centers. We then formulate it as an optimization problem that can be efficiently solved by a stochastic expectation maximization (EM) algorithm. Experiments on multiple benchmark datasets of FL show that our method outperforms several popular baseline methods. The experimental source codes are publicly available on the Github repository https://github.com/mingxuts/multi-center-fed-learning .