mlsanityData to 7 Oct 2026

Byzantine-Resilient Non-Convex Stochastic Gradient Descent

arXiv 2012.14368 ↗First posted 28 Dec 2020cs.LG cs.DC cs.DScited by 56

Not on the map. Why?

Abstract (arXiv)

We study adversary-resilient stochastic distributed optimization, in which m machines can independently compute stochastic gradients, and cooperate to jointly optimize over their local objective functions. However, an α-fraction of the machines are textitByzantine, in that they may behave in arbitrary, adversarial ways. We consider a variant of this procedure in the challenging textitnon-convex case. Our main result is a new algorithm SafeguardSGD which can provably escape saddle points and find approximate local minima of the non-convex objective. The algorithm is based on a new concentration filtering technique, and its sample and time complexity bounds match the best known theoretical bounds in the stochastic, distributed setting when no Byzantine machines are present. Our algorithm is very practical: it improves upon the performance of all prior methods when training deep neural networks, it is relatively lightweight, and it is the first method to withstand two recently-proposed Byzantine attacks.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 9; 2023: 13; 2024: 11; 2025: 13; 2026: 6.

56 papers.

Sort:

By area:

  1. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 64
  2. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  3. 2021Fault-Tolerant Federated Reinforcement Learning with Theoretical Guaranteecited by 52
  4. 2022Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guaranteescited by 30
  5. 2022Byzantine-Robust Decentralized Learning via ClippedGossipcited by 29
  6. 2021Byzantine-robust Federated Learning through Collaborative Malicious Gradient Filteringcited by 28
  7. 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneitycited by 25
  8. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  9. 2022Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teamingcited by 14
  10. 2023Byzantine-Robust Learning on Heterogeneous Data via Gradient Splittingcited by 13
  11. 2024Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updatescited by 13
  12. 2022Robust Collaborative Learning with Linear Gradient Overheadcited by 12
38 more
  1. 2023Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Ratescited by 12
  2. 2021Secure Distributed Training at Scalecited by 11
  3. 2024Adaptive Gradient Clipping for Robust Federated Learningcited by 10
  4. 2023Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneitycited by 8
  5. 2025Towards Trustworthy Federated Learning with Untrusted Participantscited by 8
  6. 2022zPROBE: Zero Peek Robustness Checks for Federated Learningcited by 6
  7. 2024Global Convergence Guarantees for Federated Policy Gradient Methods with Adversariescited by 6
  8. 2022MixTailor: Mixed Gradient Aggregation for Robust Learning Against Tailored Attackscited by 5
  9. 2024Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workerscited by 5
  10. 2024On the Byzantine-Resilience of Distillation-Based Federated Learningcited by 5
  11. 2023Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outlierscited by 4
  12. 2024A survey on secure decentralized optimization and learningcited by 4
  13. 2023FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learningcited by 3
  14. 2024Overcoming the Challenges of Batch Normalization in Federated Learningcited by 3
  15. 2024Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Trainingcited by 3
  16. 2025Bant: Byzantine Antidote via Trial Function and Trust Scorescited by 3
  17. 2025ByzFL: Research Framework for Robust Federated Learningcited by 3
  18. 2025Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participationcited by 3
  19. 2024Decentralized Federated Policy Gradient with Byzantine Fault-Tolerance and Provably Fast Convergencecited by 2
  20. 2025Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous MLcited by 2
  21. 2025Tight Stability Bounds for Robust Distributed Learning: Byzantine Failures Hurt Generalization More than Data Poisoningcited by 2
  22. 2021Stochastic Alternating Direction Method of Multipliers for Byzantine-Robust Distributed Learningcited by 1
  23. 2022FedCut: A Spectral Analysis Framework for Reliable Detection of Byzantine Colluderscited by 1
  24. 2025A Nesterov-Accelerated Byzantine-Robust Federated Learningcited by 1
  25. 2022Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Topcited by 0
  26. 2023A Robust Classification Framework for Byzantine-Resilient Stochastic Gradient Descentcited by 0
  27. 2023Flag Aggregator: Scalable Distributed Training under Failures and Augmented Losses using Convex Optimizationcited by 0
  28. 2023On the Optimal Batch Size for Byzantine-Robust Distributed Learningcited by 0
  29. 2023SABLE: Secure And Byzantine robust LEarningcited by 0
  30. 2023Byzantine-Resilient Federated PCA and Low Rank Column-wise Sensingcited by 0
  31. 2024LiD-FL: Towards List-Decodable Federated Learningcited by 0
  32. 2024FedCAP: Robust Federated Learning via Customized Aggregation and Personalizationcited by 0
  33. 2025Decoding FL Defenses: Systemization, Pitfalls, and Remediescited by 0
  34. 2025Private Aggregation for Byzantine-Resilient Heterogeneous Federated Learningcited by 0
  35. 2025Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learningcited by 0
  36. 2025ProDiGy: Proximity- and Dissimilarity-Based Byzantine-Robust Federated Learningcited by 0
  37. 2025Robust and Efficient Collaborative Learningcited by 0
  38. 2025SpectralKrum: A Spectral-Geometric Defense Against Byzantine Attacks in Federated Learningcited by 0

50 most cited of 56 All 56 (JSON)