# Robust Federated Learning by Mixture of Experts

arXiv: https://arxiv.org/abs/2104.11700 · HTML: https://mlsanity.com/paper/2104.11700/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Federated mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Placed in

- Mixture-of-experts / [Federated mixture-of-experts](https://mlsanity.com/moe/federated-mixture-of-experts/)

## Builds on (most cited first, 12 of 12)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [How To Backdoor Federated Learning](https://mlsanity.com/paper/1807.00459/) (2018)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Fair Resource Allocation in Federated Learning](https://mlsanity.com/paper/1905.10497/) (2019)
- [A Unified Theory of Decentralized SGD with Changing Topology and Local Updates](https://mlsanity.com/paper/2003.10422/) (2020)
- [Robust Aggregation for Federated Learning](https://mlsanity.com/paper/1912.13445/) (2019)
- [Can You Really Backdoor Federated Learning?](https://mlsanity.com/paper/1911.07963/) (2019)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Minibatch vs Local SGD for Heterogeneous Distributed Learning](https://mlsanity.com/paper/2006.04735/) (2020)
- [A generic framework for privacy preserving deep learning](https://mlsanity.com/paper/1811.04017/) (2018)
- [A Primal-Dual SGD Algorithm for Distributed Nonconvex Optimization](https://mlsanity.com/paper/2006.03474/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning](https://mlsanity.com/paper/2408.07057/) (2024)
- [Advances and Open Challenges in Federated Foundation Models](https://mlsanity.com/paper/2404.15381/) (2024)
- [Towards Vision Mixture of Experts for Wildlife Monitoring on the Edge](https://mlsanity.com/paper/2411.07834/) (2024)
- [Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learning](https://mlsanity.com/paper/2506.18789/) (2025)

## Abstract (arXiv)

We present a novel weighted average model based on the mixture of experts (MoE) concept to provide robustness in Federated learning (FL) against the poisoned/corrupted/outdated local models. These threats along with the non-IID nature of data sets can considerably diminish the accuracy of the FL model. Our proposed MoE-FL setup relies on the trust between users and the server where the users share a portion of their public data sets with the server. The server applies a robust aggregation method by solving the optimization problem or the Softmax method to highlight the outlier cases and to reduce their adverse effect on the FL process. Our experiments illustrate that MoE-FL outperforms the performance of the traditional aggregation approach for high rate of poisoned data from attackers.