# Byzantine-Resilient High-Dimensional Federated Learning

arXiv: https://arxiv.org/abs/2006.13041 · HTML: https://mlsanity.com/paper/2006.13041/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2020.

## Built on by (most cited first, 27 of 27)

- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Byzantine Machine Learning Made Easy by Resilient Averaging of Momentums](https://mlsanity.com/paper/2205.12173/) (2022)
- [Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guarantees](https://mlsanity.com/paper/2205.11765/) (2022)
- [On the Privacy-Robustness-Utility Trilemma in Distributed Learning](https://mlsanity.com/paper/2302.04787/) (2023)
- [Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting](https://mlsanity.com/paper/2302.06079/) (2023)
- [Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updates](https://mlsanity.com/paper/2402.12780/) (2024)
- [FRAG: Toward Federated Vector Database Management for Collaborative and Secure Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.13272/) (2024)
- [OmniLytics: A Blockchain-based Secure Data Market for Decentralized Machine Learning](https://mlsanity.com/paper/2107.05252/) (2021)
- [Certifiably Byzantine-Robust Federated Conformal Prediction](https://mlsanity.com/paper/2406.01960/) (2024)
- [Byzantine-Resilient Federated Learning at Edge](https://mlsanity.com/paper/2303.10434/) (2023)
- [MixTailor: Mixed Gradient Aggregation for Robust Learning Against Tailored Attacks](https://mlsanity.com/paper/2207.07941/) (2022)
- [Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workers](https://mlsanity.com/paper/2402.02951/) (2024)
- [Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outliers](https://mlsanity.com/paper/2312.12835/) (2023)
- [Federated Learning Resilient to Byzantine Attacks and Data Heterogeneity](https://mlsanity.com/paper/2403.13374/) (2024)
- [A simplified convergence theory for Byzantine resilient stochastic gradient descent](https://mlsanity.com/paper/2208.11879/) (2022)
- [Online Learning with Adversaries: A Differential-Inclusion Analysis](https://mlsanity.com/paper/2304.01525/) (2023)
- [Bant: Byzantine Antidote via Trial Function and Trust Scores](https://mlsanity.com/paper/2505.07614/) (2025)
- [Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation](https://mlsanity.com/paper/2509.02970/) (2025)
- [On the Relevance of Byzantine Robust Optimization Against Data Poisoning](https://mlsanity.com/paper/2405.00491/) (2024)
- [FedCut: A Spectral Analysis Framework for Reliable Detection of Byzantine Colluders](https://mlsanity.com/paper/2211.13389/) (2022)
- [Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filtering](https://mlsanity.com/paper/2409.08640/) (2024)
- [Robust Federated Personalised Mean Estimation for the Gaussian Mixture Model](https://mlsanity.com/paper/2504.19955/) (2025)
- [On the Gradient Heterogeneity Dynamics of Adversarially Robust Federated Regression](https://mlsanity.com/paper/2609.25705/) (2026)
- [A Robust Classification Framework for Byzantine-Resilient Stochastic Gradient Descent](https://mlsanity.com/paper/2301.07498/) (2023)
- [Centroid Approximation for Byzantine-Tolerant Federated Learning](https://mlsanity.com/paper/2506.15264/) (2025)
- [Byzantine-resilient federated online learning for Gaussian process regression](https://mlsanity.com/paper/2507.14021/) (2025)
- [Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds](https://mlsanity.com/paper/2604.10179/) (2026)

## Abstract (arXiv)

We study stochastic gradient descent (SGD) with local iterations in the presence of malicious/Byzantine clients, motivated by the federated learning. The clients, instead of communicating with the central server in every iteration, maintain their local models, which they update by taking several SGD iterations based on their own datasets and then communicate the net update with the server, thereby achieving communication-efficiency. Furthermore, only a subset of clients communicate with the server, and this subset may be different at different synchronization times. The Byzantine clients may collaborate and send arbitrary vectors to the server to disrupt the learning process. To combat the adversary, we employ an efficient high-dimensional robust mean estimation algorithm from Steinhardt et al.~cite[ITCS 2018]ResilienceSCV18 at the server to filter-out corrupt vectors; and to analyze the outlier-filtering procedure, we develop a novel matrix concentration result that may be of independent interest. We provide convergence analyses for strongly-convex and non-convex smooth objectives in the heterogeneous data setting, where different clients may have different local datasets, and we do not make any probabilistic assumptions on data generation. We believe that ours is the first Byzantine-resilient algorithm and analysis with local iterations. We derive our convergence results under minimal assumptions of bounded variance for SGD and bounded gradient dissimilarity (which captures heterogeneity among local datasets). We also extend our results to the case when clients compute full-batch gradients.