# Byzantine-Resilient Stochastic Gradient Descent for Distributed Learning: A Lipschitz-Inspired Coordinate-wise Median Approach

arXiv: https://arxiv.org/abs/1909.04532 · HTML: https://mlsanity.com/paper/1909.04532/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2019.

## Built on by (most cited first, 13 of 13)

- [FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMs](https://mlsanity.com/paper/2306.04959/) (2023)
- [Robust Training in High Dimensions via Block Coordinate Geometric Median Descent](https://mlsanity.com/paper/2106.08882/) (2021)
- [Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challenges](https://mlsanity.com/paper/2407.06754/) (2024)
- [Byzantine-Robust and Privacy-Preserving Framework for FedML](https://mlsanity.com/paper/2105.02295/) (2021)
- [OmniLytics: A Blockchain-based Secure Data Market for Decentralized Machine Learning](https://mlsanity.com/paper/2107.05252/) (2021)
- [A Secure Federated Learning Framework for Residential Short Term Load Forecasting](https://mlsanity.com/paper/2209.14547/) (2022)
- [Proof-of-Contribution-Based Design for Collaborative Machine Learning on Blockchain](https://mlsanity.com/paper/2302.14031/) (2023)
- [Kick Bad Guys Out! Conditionally Activated Anomaly Detection in Federated Learning with Zero-Knowledge Proof Verification](https://mlsanity.com/paper/2310.04055/) (2023)
- [Byzantine-Robust Federated Learning via Credibility Assessment on Non-IID Data](https://mlsanity.com/paper/2109.02396/) (2021)
- [FedCut: A Spectral Analysis Framework for Reliable Detection of Byzantine Colluders](https://mlsanity.com/paper/2211.13389/) (2022)
- [mathsfVISTA: Decentralized Machine Learning in Adversary Dominated Environments](https://mlsanity.com/paper/2605.07841/) (2026)
- [A Robust Classification Framework for Byzantine-Resilient Stochastic Gradient Descent](https://mlsanity.com/paper/2301.07498/) (2023)
- [GRANITE : a Byzantine-Resilient Dynamic Gossip Learning Framework](https://mlsanity.com/paper/2504.17471/) (2025)

## Abstract (arXiv)

In this work, we consider the resilience of distributed algorithms based on stochastic gradient descent (SGD) in distributed learning with potentially Byzantine attackers, who could send arbitrary information to the parameter server to disrupt the training process. Toward this end, we propose a new Lipschitz-inspired coordinate-wise median approach (LICM-SGD) to mitigate Byzantine attacks. We show that our LICM-SGD algorithm can resist up to half of the workers being Byzantine attackers, while still converging almost surely to a stationary region in non-convex settings. Also, our LICM-SGD method does not require any information about the number of attackers and the Lipschitz constant, which makes it attractive for practical implementations. Moreover, our LICM-SGD method enjoys the optimal O(md) computational time-complexity in the sense that the time-complexity is the same as that of the standard SGD under no attacks. We conduct extensive experiments to show that our LICM-SGD algorithm consistently outperforms existing methods in training multi-class logistic regression and convolutional neural networks with MNIST and CIFAR-10 datasets. In our experiments, LICM-SGD also achieves a much faster running time thanks to its low computational time-complexity.