# Robust Federated Learning in a Heterogeneous Environment

arXiv: https://arxiv.org/abs/1906.06629 · HTML: https://mlsanity.com/paper/1906.06629/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2019.

## Built on by (most cited first, 30 of 90)

- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data](https://mlsanity.com/paper/2106.05001/) (2021)
- [Multi-Center Federated Learning: Clients Clustering for Better Personalization](https://mlsanity.com/paper/2108.08647/) (2021)
- [Heterogeneity for the Win: One-Shot Federated Clustering](https://mlsanity.com/paper/2103.00697/) (2021)
- [Decentralized Federated Learning: A Survey and Perspective](https://mlsanity.com/paper/2306.01603/) (2023)
- [FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning](https://mlsanity.com/paper/2102.02514/) (2021)
- [Communication-Efficient Adaptive Federated Learning](https://mlsanity.com/paper/2205.02719/) (2022)
- [Federated Continual Learning via Knowledge Fusion: A Survey](https://mlsanity.com/paper/2312.16475/) (2023)
- [Adversarial Attacks On Multi-Agent Communication](https://mlsanity.com/paper/2101.06560/) (2021)
- [Federated Learning: A Signal Processing Perspective](https://mlsanity.com/paper/2103.17150/) (2021)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [Federated Learning under Distributed Concept Drift](https://mlsanity.com/paper/2206.00799/) (2022)
- [DYNAFED: Tackling Client Data Heterogeneity with Global Dynamics](https://mlsanity.com/paper/2211.10878/) (2022)
- [GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learning](https://mlsanity.com/paper/2108.02741/) (2021)
- [Exploiting Heterogeneity in Robust Federated Best-Arm Identification](https://mlsanity.com/paper/2109.05700/) (2021)
- [Optimizing the Collaboration Structure in Cross-Silo Federated Learning](https://mlsanity.com/paper/2306.06508/) (2023)
- [Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework](https://mlsanity.com/paper/2409.11585/) (2024)
- [A Huber Loss Minimization Approach to Byzantine Robust Federated Learning](https://mlsanity.com/paper/2308.12581/) (2023)
- [Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challenges](https://mlsanity.com/paper/2407.06754/) (2024)
- [Communication-Efficient Federated Learning for Heterogeneous Edge Devices Based on Adaptive Gradient Quantization](https://mlsanity.com/paper/2212.08272/) (2022)
- [Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting](https://mlsanity.com/paper/2302.06079/) (2023)
- [A Survey on Class Imbalance in Federated Learning](https://mlsanity.com/paper/2303.11673/) (2023)
- [Secure Byzantine-Robust Distributed Learning via Clustering](https://mlsanity.com/paper/2110.02940/) (2021)
- [An Optimal Transport Approach to Personalized Federated Learning](https://mlsanity.com/paper/2206.02468/) (2022)
- [Communication-Efficient and Personalized Federated Lottery Ticket Learning](https://mlsanity.com/paper/2104.12501/) (2021)
- [Decentralized Low-Latency Collaborative Inference via Ensembles on the Edge](https://mlsanity.com/paper/2206.03165/) (2022)
- [Dynamic Attention-based Communication-Efficient Federated Learning](https://mlsanity.com/paper/2108.05765/) (2021)
- [Co-clustering for Federated Recommender System](https://mlsanity.com/paper/2411.01690/) (2024)
- [BOBA: Byzantine-Robust Federated Learning with Label Skewness](https://mlsanity.com/paper/2208.12932/) (2022)

## Abstract (arXiv)

We study a recently proposed large-scale distributed learning paradigm, namely Federated Learning, where the worker machines are end users' own devices. Statistical and computational challenges arise in Federated Learning particularly in the presence of heterogeneous data distribution (i.e., data points on different devices belong to different distributions signifying different clusters) and Byzantine machines (i.e., machines that may behave abnormally, or even exhibit arbitrary and potentially adversarial behavior). To address the aforementioned challenges, first we propose a general statistical model for this problem which takes both the cluster structure of the users and the Byzantine machines into account. Then, leveraging the statistical model, we solve the robust heterogeneous Federated Learning problem optimally; in particular our algorithm matches the lower bound on the estimation error in dimension and the number of data points. Furthermore, as a by-product, we prove statistical guarantees for an outlier-robust clustering algorithm, which can be considered as the Lloyd algorithm with robust estimation. Finally, we show via synthetic as well as real data experiments that the estimation error obtained by our proposed algorithm is significantly better than the non-Byzantine-robust algorithms; in particular, we gain at least by 53% and 33% for synthetic and real data experiments, respectively, in typical settings.