# FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization

arXiv: https://arxiv.org/abs/2305.02894 · HTML: https://mlsanity.com/paper/2305.02894/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 4 May 2023.

## Builds on (most cited first, 28 of 28)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [See through Gradients: Image Batch Recovery via GradInversion](https://mlsanity.com/paper/2104.07586/) (2021)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Personalized Federated Learning with First Order Model Optimization](https://mlsanity.com/paper/2012.08565/) (2020)
- [Evaluating Gradient Inversion Attacks and Defenses in Federated Learning](https://mlsanity.com/paper/2112.00059/) (2021)
- [Multi-Center Federated Learning: Clients Clustering for Better Personalization](https://mlsanity.com/paper/2108.08647/) (2021)
- [Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage](https://mlsanity.com/paper/2203.15696/) (2022)
- [FedSoft: Soft Clustered Federated Learning with Proximal Local Updating](https://mlsanity.com/paper/2112.06053/) (2021)
- [On the Convergence of Clustered Federated Learning](https://mlsanity.com/paper/2202.06187/) (2022)
- [A consensus-based global optimization method for high dimensional machine learning problems](https://mlsanity.com/paper/1909.09249/) (2019)
- [A consensus-based model for global optimization and its mean-field limit](https://mlsanity.com/paper/1604.05648/) (2016)
- [Consensus-Based Optimization Methods Converge Globally](https://mlsanity.com/paper/2103.15130/) (2021)
- [Consensus-Based Optimization on the Sphere: Convergence to Global Minimizers and Machine Learning](https://mlsanity.com/paper/2001.11988/) (2020)
- [Trends in Consensus-based optimization](https://mlsanity.com/paper/2104.01383/) (2021)
- [Polarized consensus-based dynamics for optimization and sampling](https://mlsanity.com/paper/2211.05238/) (2022)
- [On the mean-field limit for the consensus-based optimization](https://mlsanity.com/paper/2105.12919/) (2021)
- [Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law](https://mlsanity.com/paper/2211.12184/) (2022)
- [Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints](https://mlsanity.com/paper/2111.02970/) (2021)
- [Consensus-Based Optimization for Saddle Point Problems](https://mlsanity.com/paper/2212.12334/) (2022)
- [A consensus-based algorithm for multi-objective optimization and its mean-field description](https://mlsanity.com/paper/2203.16384/) (2022)

## Built on by (most cited first, 7 of 7)

- [Gradient is All You Need? How Consensus-Based Optimization can be Interpreted as a Stochastic Relaxation of Gradient Descent](https://mlsanity.com/paper/2306.09778/) (2023)
- [A survey on Clustered Federated Learning: Taxonomy, Analysis and Applications](https://mlsanity.com/paper/2501.17512/) (2025)
- [Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization](https://mlsanity.com/paper/2412.02535/) (2024)
- [Heterogeneity-Aware Knowledge Sharing for Graph Federated Learning](https://mlsanity.com/paper/2601.21589/) (2026)
- [FedEGG: Federated Learning with Explicit Global Guidance](https://mlsanity.com/paper/2404.11888/) (2024)
- [Tackling Heterogeneity in Federated Learning via Variance-Reduced Boltzmann Sampling within Homogeneous Social Coalitions](https://mlsanity.com/paper/2506.02897/) (2025)
- [MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning](https://mlsanity.com/paper/2510.21093/) (2025)

## Abstract (arXiv)

Federated learning is an important framework in modern machine learning that seeks to integrate the training of learning models from multiple users, each user having their own local data set, in a way that is sensitive to data privacy and to communication loss constraints. In clustered federated learning, one assumes an additional unknown group structure among users, and the goal is to train models that are useful for each group, rather than simply training a single global model for all users. In this paper, we propose a novel solution to the problem of clustered federated learning that is inspired by ideas in consensus-based optimization (CBO). Our new CBO-type method is based on a system of interacting particles that is oblivious to group memberships. Our model is motivated by rigorous mathematical reasoning, including a mean field analysis describing the large number of particles limit of our particle system, as well as convergence guarantees for the simultaneous global optimization of general non-convex objective functions (corresponding to the loss functions of each cluster of users) in the mean-field regime. Experimental results demonstrate the efficacy of our FedCBO algorithm compared to other state-of-the-art methods and help validate our methodological and theoretical work.