# Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching

arXiv: https://arxiv.org/abs/2203.12285 · HTML: https://mlsanity.com/paper/2203.12285/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 23 Mar 2022.

## Builds on (most cited first, 28 of 28)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Deep Domain Confusion: Maximizing for Domain Invariance](https://mlsanity.com/paper/1412.3474/) (2014)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017)
- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [Federated Learning for Internet of Things: A Comprehensive Survey](https://mlsanity.com/paper/2104.07914/) (2021)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Personalized Federated Learning with First Order Model Optimization](https://mlsanity.com/paper/2012.08565/) (2020)
- [Federated Adversarial Domain Adaptation](https://mlsanity.com/paper/1911.02054/) (2019)
- [Peer-to-peer Federated Learning on Graphs](https://mlsanity.com/paper/1901.11173/) (2019)
- [Multi-source Distilling Domain Adaptation](https://mlsanity.com/paper/1911.11554/) (2019)
- [Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data](https://mlsanity.com/paper/2102.04761/) (2021)
- [Personalized and Private Peer-to-Peer Machine Learning](https://mlsanity.com/paper/1705.08435/) (2017)
- [Consensus Control for Decentralized Deep Learning](https://mlsanity.com/paper/2102.04828/) (2021)
- [Flexible Clustered Federated Learning for Client-Level Data Distribution Shift](https://mlsanity.com/paper/2108.09749/) (2021)
- [Decentralized Federated Learning via Mutual Knowledge Transfer](https://mlsanity.com/paper/2012.13063/) (2020)
- [Throughput-Optimal Topology Design for Cross-Silo Federated Learning](https://mlsanity.com/paper/2010.12229/) (2020)
- [Decentralized federated learning of deep neural networks on non-iid data](https://mlsanity.com/paper/2107.08517/) (2021)
- [Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection](https://mlsanity.com/paper/2002.09692/) (2020)
- [Optimal Rate Adaption in Federated Learning with Compressed Communications](https://mlsanity.com/paper/2112.06694/) (2021)
- [Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems](https://mlsanity.com/paper/2007.03278/) (2020)
- [Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates](https://mlsanity.com/paper/2105.04153/) (2021)
- [Ensemble Federated Adversarial Training with Non-IID data](https://mlsanity.com/paper/2110.14814/) (2021)

## Built on by (most cited first, 24 of 24)

- [Revisiting Weighted Aggregation in Federated Learning with Neural Networks](https://mlsanity.com/paper/2302.10911/) (2023)
- [No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier](https://mlsanity.com/paper/2303.10058/) (2023)
- [A Survey on Decentralized Federated Learning](https://mlsanity.com/paper/2308.04604/) (2023)
- [Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey](https://mlsanity.com/paper/2412.13840/) (2024)
- [Text-to-Model: Text-Conditioned Neural Network Diffusion for Train-Once-for-All Personalization](https://mlsanity.com/paper/2405.14132/) (2024)
- [dFLMoE: Decentralized Federated Learning via Mixture of Experts for Medical Data Analysis](https://mlsanity.com/paper/2503.10412/) (2025)
- [Edge-cloud Collaborative Learning with Federated and Centralized Features](https://mlsanity.com/paper/2304.05871/) (2023)
- [FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learning](https://mlsanity.com/paper/2311.18559/) (2023)
- [Proximity-based Self-Federated Learning](https://mlsanity.com/paper/2407.12410/) (2024)
- [ProFed: a Benchmark for Proximity-based non-IID Federated Learning](https://mlsanity.com/paper/2503.20618/) (2025)
- [Federated Source-free Domain Adaptation for Classification: Weighted Cluster Aggregation for Unlabeled Data](https://mlsanity.com/paper/2412.13757/) (2024)
- [A survey on Clustered Federated Learning: Taxonomy, Analysis and Applications](https://mlsanity.com/paper/2501.17512/) (2025)
- [OCD-FL: A Novel Communication-Efficient Peer Selection-based Decentralized Federated Learning](https://mlsanity.com/paper/2403.04037/) (2024)
- [PeerFL: A Simulator for Peer-to-Peer Federated Learning at Scale](https://mlsanity.com/paper/2405.17839/) (2024)
- [Decentralized Personalized Federated Learning](https://mlsanity.com/paper/2406.06520/) (2024)
- [Privacy-Preserved Taxi Demand Prediction System Utilizing Distributed Data](https://mlsanity.com/paper/2408.04931/) (2024)
- [On the effects of similarity metrics in decentralized deep learning under distributional shift](https://mlsanity.com/paper/2409.10720/) (2024)
- [Sparse Self-Federated Learning for Energy Efficient Cooperative Intelligence in Society 5.0](https://mlsanity.com/paper/2507.07613/) (2025)
- [FedSiKD: Clients Similarity and Knowledge Distillation: Addressing Non-i.i.d. and Constraints in Federated Learning](https://mlsanity.com/paper/2402.09095/) (2024)
- [FedGuCci: Making Local Models More Connected in Landscape for Federated Learning](https://mlsanity.com/paper/2402.18949/) (2024)
- [P4: Towards private, personalized, and Peer-to-Peer learning](https://mlsanity.com/paper/2405.17697/) (2024)
- [Sequence-to-sequence models in peer-to-peer learning: A practical application](https://mlsanity.com/paper/2406.02565/) (2024)
- [Query-based Knowledge Transfer for Heterogeneous Learning Environments](https://mlsanity.com/paper/2504.09205/) (2025)
- [Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learning](https://mlsanity.com/paper/2506.20413/) (2025)

## Abstract (arXiv)

In federated learning (FL), clients may have diverse objectives, and merging all clients' knowledge into one global model will cause negative transfer to local performance. Thus, clustered FL is proposed to group similar clients into clusters and maintain several global models. In the literature, centralized clustered FL algorithms require the assumption of the number of clusters and hence are not effective enough to explore the latent relationships among clients. In this paper, without assuming the number of clusters, we propose a peer-to-peer (P2P) FL algorithm named PANM. In PANM, clients communicate with peers to adaptively form an effective clustered topology. Specifically, we present two novel metrics for measuring client similarity and a two-stage neighbor matching algorithm based Monte Carlo method and Expectation Maximization under the Gaussian Mixture Model assumption. We have conducted theoretical analyses of PANM on the probability of neighbor estimation and the error gap to the clustered optimum. We have also implemented extensive experiments under both synthetic and real-world clustered heterogeneity. Theoretical analysis and empirical experiments show that the proposed algorithm is superior to the P2P FL counterparts, and it achieves better performance than the centralized cluster FL method. PANM is effective even under extremely low communication budgets.