# Byzantine-resilient Decentralized Stochastic Gradient Descent

arXiv: https://arxiv.org/abs/2002.08569 · HTML: https://mlsanity.com/paper/2002.08569/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2020.

## Built on by (most cited first, 28 of 28)

- [Byzantine-Robust Decentralized Federated Learning](https://mlsanity.com/paper/2406.10416/) (2024)
- [Byzantine-Robust Decentralized Learning via ClippedGossip](https://mlsanity.com/paper/2202.01545/) (2022)
- [Federated Model Heterogeneous Matryoshka Representation Learning](https://mlsanity.com/paper/2406.00488/) (2024)
- [Robust Collaborative Learning with Linear Gradient Overhead](https://mlsanity.com/paper/2209.10931/) (2022)
- [Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learning](https://mlsanity.com/paper/2410.19321/) (2024)
- [SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learning](https://mlsanity.com/paper/2405.09394/) (2024)
- [Towards Trustworthy AI-Empowered Real-Time Bidding for Online Advertisement Auctioning](https://mlsanity.com/paper/2210.07770/) (2022)
- [Privacy-Preserving Aggregation for Decentralized Learning with Byzantine-Robustness](https://mlsanity.com/paper/2404.17970/) (2024)
- [A survey on secure decentralized optimization and learning](https://mlsanity.com/paper/2408.08628/) (2024)
- [From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challenges](https://mlsanity.com/paper/2503.07505/) (2025)
- [Competitive Advantage Attacks to Decentralized Federated Learning](https://mlsanity.com/paper/2310.13862/) (2023)
- [On the Conflict of Robustness and Learning in Collaborative Machine Learning](https://mlsanity.com/paper/2402.13700/) (2024)
- [Screen Before You Fetch: Compressed Byzantine Screening for Decentralized Learning on the Edge-Cloud Continuum](https://mlsanity.com/paper/2510.07922/) (2025)
- [A Secure and Private Distributed Bayesian Federated Learning Design](https://mlsanity.com/paper/2602.20003/) (2026)
- [Automated Byzantine-Resilient Clustered Decentralized Federated Learning for Battery Intelligence in Connected EVs](https://mlsanity.com/paper/2605.21115/) (2026)
- [Model-Consistent Byzantine-Resilient Decentralized Federated Learning for Collaborative Missions](https://mlsanity.com/paper/2608.22552/) (2026)
- [Byzantine-Resilient Learning Beyond Gradients: Distributing Evolutionary Search](https://mlsanity.com/paper/2304.13540/) (2023)
- [MultiConfederated Learning: Inclusive Non-IID Data handling with Decentralized Federated Learning](https://mlsanity.com/paper/2404.13421/) (2024)
- [VALID: a Validated Algorithm for Learning in Decentralized Networks with Possible Adversarial Presence](https://mlsanity.com/paper/2405.07316/) (2024)
- [Resilient Peer-to-peer Learning based on Adaptive Aggregation](https://mlsanity.com/paper/2501.04610/) (2025)
- [Online Decentralized Federated Multi-task Learning With Trustworthiness in Cyber-Physical Systems](https://mlsanity.com/paper/2509.00992/) (2025)
- [Adaptive Decentralized Federated Learning for Robust Optimization](https://mlsanity.com/paper/2512.02852/) (2025)
- [Evidential Trust-Aware Model Personalization in Decentralized Federated Learning for Wearable IoT](https://mlsanity.com/paper/2512.19131/) (2025)
- [Priority-Aware Learning-Unlearning Correction for Dynamic Decentralized LoRA Fine-Tuning](https://mlsanity.com/paper/2606.22878/) (2026)
- [BackDFL: A Unified Benchmark For Backdoor Attacks and Defenses In Decentralized Federated Learning](https://mlsanity.com/paper/2608.21137/) (2026)
- [Optimizing Byzantine Node Placement in Decentralized Federated Learning](https://mlsanity.com/paper/2609.01495/) (2026)
- [Robust Decentralized Federated Distillation via Multi-Modality Knowledge Collaboration](https://mlsanity.com/paper/2609.07230/) (2026)
- [Fully Byzantine-Resilient Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2609.25701/) (2026)

## Abstract (arXiv)

Decentralized learning has gained great popularity to improve learning efficiency and preserve data privacy. Each computing node makes equal contribution to collaboratively learn a Deep Learning model. The elimination of centralized Parameter Servers (PS) can effectively address many issues such as privacy, performance bottleneck and single-point-failure. However, how to achieve Byzantine Fault Tolerance in decentralized learning systems is rarely explored, although this problem has been extensively studied in centralized systems. In this paper, we present an in-depth study towards the Byzantine resilience of decentralized learning systems with two contributions. First, from the adversarial perspective, we theoretically illustrate that Byzantine attacks are more dangerous and feasible in decentralized learning systems: even one malicious participant can arbitrarily alter the models of other participants by sending carefully crafted updates to its neighbors. Second, from the defense perspective, we propose UBAR, a novel algorithm to enhance decentralized learning with Byzantine Fault Tolerance. Specifically, UBAR provides a Uniform Byzantine-resilient Aggregation Rule for benign nodes to select the useful parameter updates and filter out the malicious ones in each training iteration. It guarantees that each benign node in a decentralized system can train a correct model under very strong Byzantine attacks with an arbitrary number of faulty nodes. We conduct extensive experiments on standard image classification tasks and the results indicate that UBAR can effectively defeat both simple and sophisticated Byzantine attacks with higher performance efficiency than existing solutions.