mlsanityData to 7 Oct 2026

Learning from History for Byzantine Robust Optimization

arXiv 2012.10333 ↗First posted 18 Dec 2020cs.LG cs.DC math.OCcited by 137

Not on the map. Why?

Abstract (arXiv)

Byzantine robustness has received significant attention recently given its importance for distributed and federated learning. In spite of this, we identify severe flaws in existing algorithms even when the data across the participants is identically distributed. First, we show realistic examples where current state of the art robust aggregation rules fail to converge even in the absence of any Byzantine attackers. Secondly, we prove that even if the aggregation rules may succeed in limiting the influence of the attackers in a single round, the attackers can couple their attacks across time eventually leading to divergence. To address these issues, we present two surprisingly simple strategies: a new robust iterative clipping procedure, and incorporating worker momentum to overcome time-coupled attacks. This is the first provably robust method for the standard stochastic optimization setting. Our code is open sourced at https://github.com/epfml/byzantine-robust-optimizer.

Key facts. Cited by 137 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 20; 2023: 25; 2024: 28; 2025: 30; 2026: 22.

137 papers.

Sort:

By area:

  1. 2021A Field Guide to Federated Optimizationcited by 312
  2. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 64
  3. 2021LightSecAgg: a Lightweight and Versatile Design for Secure Aggregation in Federated Learningcited by 63
  4. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  5. 2023An Experimental Study of Byzantine-Robust Aggregation Schemes in Federated Learningcited by 32
  6. 2022Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guaranteescited by 30
  7. 2022Byzantine-Robust Decentralized Learning via ClippedGossipcited by 29
  8. 2021Byzantine-robust Federated Learning through Collaborative Malicious Gradient Filteringcited by 28
  9. 2022On the Impossible Safety of Large AI Modelscited by 27
  10. 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneitycited by 25
  11. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  12. 2022On the (In)security of Peer-to-Peer Decentralized Machine Learningcited by 23
38 more
  1. 2023FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMscited by 19
  2. 2022An Equivalence Between Data Poisoning and Byzantine Gradient Attackscited by 16
  3. 2024Poisoning Federated Recommender Systems with Fake Userscited by 15
  4. 2023Byzantine-Robust Learning on Heterogeneous Data via Gradient Splittingcited by 13
  5. 2024Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updatescited by 13
  6. 2021BROADCAST: Reducing Both Stochastic and Compression Noise to Robustify Communication-Efficient Federated Learningcited by 12
  7. 2022Bridging Differential Privacy and Byzantine-Robustness via Model Aggregationcited by 12
  8. 2022Robust Collaborative Learning with Linear Gradient Overheadcited by 12
  9. 2023Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Ratescited by 12
  10. 2024Task Arithmetic Through The Lens Of One-Shot Federated Learningcited by 12
  11. 2025Sign Operator for Coping with Heavy-Tailed Noise in Non-Convex Optimization: High Probability Bounds Under (L0, L1)-Smoothnesscited by 12
  12. 2021Secure Distributed Training at Scalecited by 11
  13. 2022Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregationcited by 11
  14. 2023Provably Personalized and Robust Federated Learningcited by 11
  15. 2023Clip21: Error Feedback for Gradient Clippingcited by 10
  16. 2024Adaptive Gradient Clipping for Robust Federated Learningcited by 10
  17. 2023Denial-of-Service or Fine-Grained Control: Towards Flexible Model Poisoning Attacks on Federated Learningcited by 9
  18. 2021Byzantine-robust Federated Learning through Spatial-temporal Analysis of Local Model Updatescited by 8
  19. 2021Linear Speedup in Personalized Collaborative Learningcited by 8
  20. 2022Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learningcited by 8
  21. 2023Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneitycited by 8
  22. 2023Byzantine-Robust Distributed Online Learning: Taming Adversarial Participants in An Adversarial Environmentcited by 8
  23. 2024Unified Breakdown Analysis for Byzantine Robust Gossipcited by 8
  24. 2025Towards Trustworthy Federated Learning with Untrusted Participantscited by 8
  25. 2021Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancycited by 7
  26. 2022zPROBE: Zero Peek Robustness Checks for Federated Learningcited by 6
  27. 2023Can Decentralized Learning be more robust than Federated Learning?cited by 6
  28. 2023Byzantine-Resilient Federated Learning at Edgecited by 6
  29. 2022Training Transformers Togethercited by 5
  30. 2022MixTailor: Mixed Gradient Aggregation for Robust Learning Against Tailored Attackscited by 5
  31. 2023Federated Learning in the Presence of Adversarial Client Unavailabilitycited by 5
  32. 2024Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workerscited by 5
  33. 2024On the Byzantine-Resilience of Distillation-Based Federated Learningcited by 5
  34. 2026Enhancing LLM Training via Spectral Clippingcited by 5
  35. 2022Collaborative Linear Bandits with Adversarial Agents: Near-Optimal Regret Boundscited by 4
  36. 2022Distributed Non-Convex Optimization with One-Bit Compressors on Heterogeneous Data: Efficient and Resilient Algorithmscited by 4
  37. 2023Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outlierscited by 4
  38. 2024Privacy-Preserving Aggregation for Decentralized Learning with Byzantine-Robustnesscited by 4

50 most cited of 137 All 137 (JSON)