mlsanityData to 7 Oct 2026

Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning

arXiv 1806.05358 ↗First posted 14 Jun 2018cs.LG cs.CR cs.DCcited by 32

Not on the map. Why?

Abstract (arXiv)

We study robust distributed learning that involves minimizing a non-convex loss function with saddle points. We consider the Byzantine setting where some worker machines have abnormal or even arbitrary and adversarial behavior. In this setting, the Byzantine machines may create fake local minima near a saddle point that is far away from any true local minimum, even when robust gradient estimators are used. We develop ByzantinePGD, a robust first-order algorithm that can provably escape saddle points and fake local minima, and converge to an approximate true local minimizer with low iteration complexity. As a by-product, we give a simpler algorithm and analysis for escaping saddle points in the usual non-Byzantine setting. We further discuss three robust gradient estimators that can be used in ByzantinePGD, including median, trimmed mean, and iterative filtering. We characterize their performance in concrete statistical settings, and argue for their near-optimality in low and high dimensional regimes.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 10; 2023: 4; 2024: 6; 2025: 4.

32 papers.

Sort:

  1. 2021A Field Guide to Federated Optimizationcited by 312
  2. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  3. 2021Provably Secure Federated Learning against Malicious Clientscited by 60
  4. 2022Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guaranteescited by 30
  5. 2022Byzantine-Robust Decentralized Learning via ClippedGossipcited by 29
  6. 2022Streaming Algorithms for High-Dimensional Robust Statisticscited by 21
  7. 2021Variance Reduced Median-of-Means Estimator for Byzantine-Robust Distributed Inferencecited by 11
  8. 2021Robust Distributed Optimization With Randomly Corrupted Gradientscited by 6
  9. 2023Can Decentralized Learning be more robust than Federated Learning?cited by 6
  10. 2021Robust Federated Learning via Over-The-Air Computationcited by 5
  11. 2022SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learningcited by 5
  12. 2022MixTailor: Mixed Gradient Aggregation for Robust Learning Against Tailored Attackscited by 5
20 more
  1. 2022Over-The-Air Federated Learning under Byzantine Attackscited by 4
  2. 2023Secure and Verifiable Data Collaboration with Low-Cost Zero-Knowledge Proofscited by 4
  3. 2021Escaping Saddle Points in Distributed Newton's Method with Communication Efficiency and Byzantine Resiliencecited by 3
  4. 2022A simplified convergence theory for Byzantine resilient stochastic gradient descentcited by 3
  5. 2023FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learningcited by 3
  6. 2024BadSampler: Harnessing the Power of Catastrophic Forgetting to Poison Byzantine-robust Federated Learningcited by 3
  7. 2025Second-Order Convergence in Private Stochastic Non-Convex Optimizationcited by 3
  8. 2022An Improved Algorithm for Clustered Federated Learningcited by 2
  9. 2024Robust Second-Order Nonconvex Optimization and Its Application to Low Rank Matrix Sensingcited by 2
  10. 2024Robust Gradient Descent for Phase Retrievalcited by 2
  11. 2021Stochastic Alternating Direction Method of Multipliers for Byzantine-Robust Distributed Learningcited by 1
  12. 2021Aspis: Robust Detection for Distributed Learningcited by 1
  13. 2024On the Conflict of Robustness and Learning in Collaborative Machine Learningcited by 1
  14. 2024Byzantine-tolerant distributed learning of finite mixture modelscited by 1
  15. 2022Detection and Mitigation of Byzantine Attacks in Distributed Trainingcited by 0
  16. 2023On the Optimal Batch Size for Byzantine-Robust Distributed Learningcited by 0
  17. 2024Distributed quasi-Newton robust estimation under differential privacycited by 0
  18. 2025Federated Independent Component Analysis via Spectral Alignment and Robust Aggregationcited by 0
  19. 2025CoSIFL: Collaborative Secure and Incentivized Federated Learning with Differential Privacycited by 0
  20. 2025Adaptive Decentralized Federated Learning for Robust Optimizationcited by 0