# MultiConfederated Learning: Inclusive Non-IID Data handling with Decentralized Federated Learning

arXiv: https://arxiv.org/abs/2404.13421 · HTML: https://mlsanity.com/paper/2404.13421/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 20 Apr 2024.

## Builds on (most cited first, 21 of 21)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [Three Approaches for Personalization with Applications to Federated Learning](https://mlsanity.com/paper/2002.10619/) (2020)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Applied Federated Learning: Improving Google Keyboard Query Suggestions](https://mlsanity.com/paper/1812.02903/) (2018)
- [Federated Evaluation of On-device Personalization](https://mlsanity.com/paper/1910.10252/) (2019)
- [Overcoming Forgetting in Federated Learning on Non-IID Data](https://mlsanity.com/paper/1910.07796/) (2019)
- [BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning](https://mlsanity.com/paper/1905.06731/) (2019)
- [A Blockchain-based Decentralized Federated Learning Framework with Committee Consensus](https://mlsanity.com/paper/2004.00773/) (2020)
- [When Machine Learning Meets Privacy: A Survey and Outlook](https://mlsanity.com/paper/2011.11819/) (2020)
- [Privacy-Preserving Machine Learning: Methods, Challenges and Directions](https://mlsanity.com/paper/2108.04417/) (2021)
- [Byzantine-resilient Decentralized Stochastic Gradient Descent](https://mlsanity.com/paper/2002.08569/) (2020)
- [FedFMC: Sequential Efficient Federated Learning on Non-iid Data](https://mlsanity.com/paper/2006.10937/) (2020)
- [FedCC: Robust Federated Learning against Model Poisoning Attacks](https://mlsanity.com/paper/2212.01976/) (2022)
- [Implicit Model Specialization through DAG-based Decentralized Federated Learning](https://mlsanity.com/paper/2111.01257/) (2021)
- [Weight Divergence Driven Divide-and-Conquer Approach for Optimal Federated Learning from non-IID Data](https://mlsanity.com/paper/2106.14503/) (2021)

## Abstract (arXiv)

Federated Learning (FL) has emerged as a prominent privacy-preserving technique for enabling use cases like confidential clinical machine learning. FL operates by aggregating models trained by remote devices which owns the data. Thus, FL enables the training of powerful global models using crowd-sourced data from a large number of learners, without compromising their privacy. However, the aggregating server is a single point of failure when generating the global model. Moreover, the performance of the model suffers when the data is not independent and identically distributed (non-IID data) on all remote devices. This leads to vastly different models being aggregated, which can reduce the performance by as much as 50% in certain scenarios. In this paper, we seek to address the aforementioned issues while retaining the benefits of FL. We propose MultiConfederated Learning: a decentralized FL framework which is designed to handle non-IID data. Unlike traditional FL, MultiConfederated Learning will maintain multiple models in parallel (instead of a single global model) to help with convergence when the data is non-IID. With the help of transfer learning, learners can converge to fewer models. In order to increase adaptability, learners are allowed to choose which updates to aggregate from their peers.