# FedSoft: Soft Clustered Federated Learning with Proximal Local Updating

arXiv: https://arxiv.org/abs/2112.06053 · HTML: https://mlsanity.com/paper/2112.06053/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 11 Dec 2021.

## Builds on (most cited first, 18 of 18)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Personalized Cross-Silo Federated Learning on Non-IID Data](https://mlsanity.com/paper/2007.03797/) (2020)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Federated Multi-Task Learning under a Mixture of Distributions](https://mlsanity.com/paper/2108.10252/) (2021)
- [One-Shot Federated Learning](https://mlsanity.com/paper/1902.11175/) (2019)
- [Multi-Center Federated Learning: Clients Clustering for Better Personalization](https://mlsanity.com/paper/2108.08647/) (2021)
- [Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Records](https://mlsanity.com/paper/1903.09296/) (2019)
- [Collaborative Federated Learning For Healthcare: Multi-Modal COVID-19 Diagnosis at the Edge](https://mlsanity.com/paper/2101.07511/) (2021)
- [An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Models](https://mlsanity.com/paper/1909.06678/) (2019)

## Built on by (most cited first, 30 of 46)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [DELTA: Diverse Client Sampling for Fasting Federated Learning](https://mlsanity.com/paper/2205.13925/) (2022)
- [FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentation](https://mlsanity.com/paper/2205.14900/) (2022)
- [FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering](https://mlsanity.com/paper/2301.12379/) (2023)
- [Exploiting Label Skews in Federated Learning with Model Concatenation](https://mlsanity.com/paper/2312.06290/) (2023)
- [Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations](https://mlsanity.com/paper/2405.09839/) (2024)
- [Co-clustering for Federated Recommender System](https://mlsanity.com/paper/2411.01690/) (2024)
- [A Comprehensive Survey of Federated Transfer Learning: Challenges, Methods and Applications](https://mlsanity.com/paper/2403.01387/) (2024)
- [FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization](https://mlsanity.com/paper/2305.02894/) (2023)
- [FOCUS: Fairness via Agent-Awareness for Federated Learning on Heterogeneous Data](https://mlsanity.com/paper/2207.10265/) (2022)
- [Federated Learning of Shareable Bases for Personalization-Friendly Image Classification](https://mlsanity.com/paper/2304.07882/) (2023)
- [FedSPD: A Soft-clustering Approach for Personalized Decentralized Federated Learning](https://mlsanity.com/paper/2410.18862/) (2024)
- [Differentially Private Clustered Federated Learning](https://mlsanity.com/paper/2405.19272/) (2024)
- [Combating Client Dropout in Federated Learning via Friend Model Substitution](https://mlsanity.com/paper/2205.13222/) (2022)
- [Enhancing Clustered Federated Learning: Integration of Strategies and Improved Methodologies](https://mlsanity.com/paper/2310.05397/) (2023)
- [Fair Concurrent Training of Multiple Models in Federated Learning](https://mlsanity.com/paper/2404.13841/) (2024)
- [A Bayesian Framework for Clustered Federated Learning](https://mlsanity.com/paper/2410.15473/) (2024)
- [An Improved Algorithm for Clustered Federated Learning](https://mlsanity.com/paper/2210.11538/) (2022)
- [Federated Two Stage Decoupling With Adaptive Personalization Layers](https://mlsanity.com/paper/2308.15821/) (2023)
- [A survey on Clustered Federated Learning: Taxonomy, Analysis and Applications](https://mlsanity.com/paper/2501.17512/) (2025)
- [Redefining Clustered Federated Learning for System Identification: The Path of ClusterCraft](https://mlsanity.com/paper/2505.16857/) (2025)
- [Operator-Theoretic Framework for Gradient-Free Federated Learning](https://mlsanity.com/paper/2512.01025/) (2025)
- [Few-for-Many Personalized Federated Learning](https://mlsanity.com/paper/2603.11992/) (2026)
- [REPA: Client Clustering without Training and Data Labels for Improved Federated Learning in Non-IID Settings](https://mlsanity.com/paper/2309.14088/) (2023)
- [Federated Cross-Training Learners for Robust Generalization under Data Heterogeneity](https://mlsanity.com/paper/2405.20046/) (2024)
- [TPFL: Tsetlin-Personalized Federated Learning with Confidence-Based Clustering](https://mlsanity.com/paper/2409.10392/) (2024)
- [Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization](https://mlsanity.com/paper/2412.02535/) (2024)
- [MMiC: Mitigating Modality Incompleteness in Clustered Federated Learning](https://mlsanity.com/paper/2505.06911/) (2025)
- [Distilling A Universal Expert from Clustered Federated Learning](https://mlsanity.com/paper/2506.20285/) (2025)

## Abstract (arXiv)

Traditionally, clustered federated learning groups clients with the same data distribution into a cluster, so that every client is uniquely associated with one data distribution and helps train a model for this distribution. We relax this hard association assumption to soft clustered federated learning, which allows every local dataset to follow a mixture of multiple source distributions. We propose FedSoft, which trains both locally personalized models and high-quality cluster models in this setting. FedSoft limits client workload by using proximal updates to require the completion of only one optimization task from a subset of clients in every communication round. We show, analytically and empirically, that FedSoft effectively exploits similarities between the source distributions to learn personalized and cluster models that perform well.