mlsanityData to 14 Oct 2026

A simplified convergence theory for Byzantine resilient stochastic gradient descent

arXiv 2208.11879 ↗First posted 25 Aug 2022cs.LG math.OCcited by 3

Not on the map. Why?

Abstract (arXiv)

In distributed learning, a central server trains a model according to updates provided by nodes holding local data samples. In the presence of one or more malicious servers sending incorrect information (a Byzantine adversary), standard algorithms for model training such as stochastic gradient descent (SGD) fail to converge. In this paper, we present a simplified convergence theory for the generic Byzantine Resilient SGD method originally proposed by Blanchard et al. [NeurIPS 2017]. Compared to the existing analysis, we shown convergence to a stationary point in expectation under standard assumptions on the (possibly nonconvex) objective function and flexible assumptions on the stochastic gradients.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 25 Aug 2022.

Builds on

19 of 31 references.

References · 19

  1. 2019Advances and Open Problems in Federated Learningcited by 3,046
  2. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  3. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  4. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  5. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  6. 2018Analyzing Federated Learning through an Adversarial Lenscited by 351
  7. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  8. 2019Robust Aggregation for Federated Learningcited by 317
  9. 2018Beyond Data and Model Parallelism for Deep Neural Networkscited by 179
  10. 2020Better Theory for SGD in the Nonconvex Worldcited by 158
  11. 2017Revisiting Distributed Synchronous SGDcited by 142
  12. 2018Generalized Byzantine-tolerant SGDcited by 95
7 more
  1. 2019The Complexity of Finding Stationary Points with Stochastic Gradient Descentcited by 35
  2. 2018Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learningcited by 32
  3. 2020A Unified Analysis of Stochastic Gradient Methods for Nonconvex Federated Optimizationcited by 28
  4. 2020Byzantine-Resilient High-Dimensional Federated Learningcited by 27
  5. 2020Byzantine-Resilient SGD in High Dimensions on Heterogeneous Datacited by 17
  6. 2019Distributed SGD Generalizes Well Under Asynchronycited by 4
  7. 2021A Distributed SGD Algorithm with Global Sketching for Deep Learning Training Accelerationcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024On the Convergence of Federated Learning Algorithms without Data Similaritycited by 2
  2. 2025Bilevel Models for Adversarial Learning and A Case Studycited by 0
  3. 2025Bilevel Learning via Inexact Stochastic Gradient Descentcited by 0