# FedSPD: A Soft-clustering Approach for Personalized Decentralized Federated Learning

arXiv: https://arxiv.org/abs/2410.18862 · HTML: https://mlsanity.com/paper/2410.18862/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 24 Oct 2024.

## Builds on (most cited first, 30 of 30)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Ditto: Fair and Robust Federated Learning Through Personalization](https://mlsanity.com/paper/2012.04221/) (2020)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [Federated Learning with Differential Privacy: Algorithms and Performance Analysis](https://mlsanity.com/paper/1911.00222/) (2019)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent](https://mlsanity.com/paper/1705.09056/) (2017)
- [A Unified Theory of Decentralized SGD with Changing Topology and Local Updates](https://mlsanity.com/paper/2003.10422/) (2020)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Personalized Federated Learning: A Meta-Learning Approach](https://mlsanity.com/paper/2002.07948/) (2020)
- [Federated Learning for Internet of Things: A Comprehensive Survey](https://mlsanity.com/paper/2104.07914/) (2021)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Federated Multi-Task Learning under a Mixture of Distributions](https://mlsanity.com/paper/2108.10252/) (2021)
- [On the Convergence of Decentralized Gradient Descent](https://mlsanity.com/paper/1310.7063/) (2013)
- [Stochastic Gradient Push for Distributed Deep Learning](https://mlsanity.com/paper/1811.10792/) (2018)
- [Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges](https://mlsanity.com/paper/2211.08413/) (2022)
- [Distributed optimization over time-varying directed graphs](https://mlsanity.com/paper/1303.2289/) (2013)
- [An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guarantee](https://mlsanity.com/paper/2011.01783/) (2020)
- [FedGroup: Efficient Clustered Federated Learning via Decomposed Data-Driven Measure](https://mlsanity.com/paper/2010.06870/) (2020)
- [FedSoft: Soft Clustered Federated Learning with Proximal Local Updating](https://mlsanity.com/paper/2112.06053/) (2021)
- [Personalization of End-to-end Speech Recognition On Mobile Devices For Named Entities](https://mlsanity.com/paper/1912.09251/) (2019)
- [Personalized Federated Learning under Mixture of Distributions](https://mlsanity.com/paper/2305.01068/) (2023)
- [Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs](https://mlsanity.com/paper/1406.2075/) (2014)
- [Personalized Decentralized Federated Learning with Knowledge Distillation](https://mlsanity.com/paper/2302.12156/) (2023)
- [Multi-task Learning for Aggregated Data using Gaussian Processes](https://mlsanity.com/paper/1906.09412/) (2019)
- [Decentralized Personalized Federated Learning: Lower Bounds and Optimal Algorithm for All Personalization Modes](https://mlsanity.com/paper/2107.07190/) (2021)
- [Decentralized Consensus Algorithm with Delayed and Stochastic Gradients](https://mlsanity.com/paper/1604.05649/) (2016)
- [Personalized Federated Learning of Driver Prediction Models for Autonomous Driving](https://mlsanity.com/paper/2112.00956/) (2021)
- [A Newton Tracking Algorithm with Exact Linear Convergence Rate for Decentralized Consensus Optimization](https://mlsanity.com/paper/2008.10157/) (2020)

## Built on by (most cited first, 5 of 5)

- [pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup](https://mlsanity.com/paper/2501.11002/) (2025)
- [DFCA: Decentralized Federated Clustering Algorithm](https://mlsanity.com/paper/2510.15300/) (2025)
- [TallyTrain: Communication-Efficient Federated Distillation](https://mlsanity.com/paper/2607.00173/) (2026)
- [Federated Lightweight Fine-Tuning](https://mlsanity.com/paper/2607.18343/) (2026)
- [Data-Topology Compatibility Enables Personalization in Decentralized Learning](https://mlsanity.com/paper/2610.15126/) (2026)

## Abstract (arXiv)

Federated learning has recently gained popularity as a framework for distributed clients to collaboratively train a machine learning model using local data. While traditional federated learning relies on a central server for model aggregation, recent advancements adopt a decentralized framework, enabling direct model exchange between clients and eliminating the single point of failure. However, existing decentralized frameworks often assume all clients train a shared model. Personalizing each client's model can enhance performance, especially with heterogeneous client data distributions. We propose FedSPD, an efficient personalized federated learning algorithm for the decentralized setting, and show that it learns accurate models even in low-connectivity networks. To provide theoretical guarantees on convergence, we introduce a clustering-based framework that enables consensus on models for distinct data clusters while personalizing to unique mixtures of these clusters at different clients. This flexibility, allowing selective model updates based on data distribution, substantially reduces communication costs compared to prior work on personalized federated learning in decentralized settings. Experimental results on real-world datasets show that FedSPD outperforms multiple decentralized variants of personalized federated learning algorithms, especially in scenarios with low-connectivity networks.