# Byzantine-Robust Variance-Reduced Federated Learning over Distributed Non-i.i.d. Data

arXiv: https://arxiv.org/abs/2009.08161 · HTML: https://mlsanity.com/paper/2009.08161/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 8 of 8)

- [Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting](https://mlsanity.com/paper/2302.06079/) (2023)
- [Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions](https://mlsanity.com/paper/2411.12377/) (2024)
- [BOBA: Byzantine-Robust Federated Learning with Label Skewness](https://mlsanity.com/paper/2208.12932/) (2022)
- [Over-The-Air Federated Learning under Byzantine Attacks](https://mlsanity.com/paper/2205.02949/) (2022)
- [Stochastic Alternating Direction Method of Multipliers for Byzantine-Robust Distributed Learning](https://mlsanity.com/paper/2106.06891/) (2021)
- [Aggressive or Imperceptible, or Both: Network Pruning Assisted Hybrid Byzantines in Federated Learning](https://mlsanity.com/paper/2404.06230/) (2024)
- [Fed-Credit: Robust Federated Learning with Credibility Management](https://mlsanity.com/paper/2405.11758/) (2024)
- [Coded Robust Aggregation for Distributed Learning under Byzantine Attacks](https://mlsanity.com/paper/2506.01989/) (2025)

## Abstract (arXiv)

We consider the federated learning problem where data on workers are not independent and identically distributed (i.i.d.). During the learning process, an unknown number of Byzantine workers may send malicious messages to the central node, leading to remarkable learning error. Most of the Byzantine-robust methods address this issue by using robust aggregation rules to aggregate the received messages, but rely on the assumption that all the regular workers have i.i.d. data, which is not the case in many federated learning applications. In light of the significance of reducing stochastic gradient noise for mitigating the effect of Byzantine attacks, we use a resampling strategy to reduce the impact of both inner variation (that describes the sample heterogeneity on every regular worker) and outer variation (that describes the sample heterogeneity among the regular workers), along with a stochastic average gradient algorithm to gradually eliminate the inner variation. The variance-reduced messages are then aggregated with a robust geometric median operator. We prove that the proposed method reaches a neighborhood of the optimal solution at a linear convergence rate and the learning error is determined by the number of Byzantine workers. Numerical experiments corroborate the theoretical results and show that the proposed method outperforms the state-of-the-arts in the non-i.i.d. setting.