mlsanityData to 14 Oct 2026

Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workers

arXiv 2402.02951 ↗First posted 5 Feb 2024cs.LG cs.DC stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

Byzantine-robust learning has emerged as a prominent fault-tolerant distributed machine learning framework. However, most techniques focus on the static setting, wherein the identity of Byzantine workers remains unchanged throughout the learning process. This assumption fails to capture real-world dynamic Byzantine behaviors, which may include intermittent malfunctions or targeted, time-limited attacks. Addressing this limitation, we propose DynaBRO -- a new method capable of withstanding any sub-linear number of identity changes across rounds. Specifically, when the number of such changes is O(√T) (where T is the total number of training rounds), DynaBRO nearly matches the state-of-the-art asymptotic convergence rate of the static setting. Our method utilizes a multi-level Monte Carlo (MLMC) gradient estimation technique applied at the server to robustly aggregated worker updates. By additionally leveraging an adaptive learning rate, we circumvent the need for prior knowledge of the fraction of Byzantine workers.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 5 Feb 2024.

Builds on

36 of 54 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2019Towards Federated Learning at Scale: System Designcited by 832
  4. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  5. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  6. 2019Robust Aggregation for Federated Learningcited by 317
  7. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
  8. 2019A Survey on Distributed Machine Learningcited by 188
  9. 2018AdaGrad stepsizes: Sharp convergence over nonconvex landscapescited by 182
  10. 2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
  11. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 141
  12. 2020Learning from History for Byzantine Robust Optimizationcited by 137
24 more
  1. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 128
  2. 2020MetaPoison: Practical General-purpose Clean-label Data Poisoningcited by 120
  3. 2020Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attackscited by 116
  4. 2020On the Convergence of SGD with Biased Gradientscited by 83
  5. 2018Byzantine Stochastic Gradient Descentcited by 81
  6. 2019Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attackscited by 79
  7. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 65
  8. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  9. 2022The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variancecited by 61
  10. 2021Distributed Deep Learning in Open Collaborationscited by 58
  11. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 57
  12. 2018Online Adaptive Methods, Universality and Accelerationcited by 51
  13. 2020Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Expertscited by 43
  14. 2022High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsizecited by 40
  15. 2017Online to Offline Conversions, Universality and Adaptive Minibatch Sizescited by 39
  16. 2022Adapting to Mixing Time in Stochastic Optimization with Markovian Datacited by 36
  17. 2023SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variancecited by 30
  18. 2020Byzantine-Resilient High-Dimensional Federated Learningcited by 27
  19. 2021Stochastic Bias-Reduced Gradient Methodscited by 25
  20. 2023Beyond Exponentially Fast Mixing in Average-Reward Reinforcement Learning via Multi-Level Monte Carlo Actor-Criticcited by 18
  21. 2023First Order Methods with Markovian Noise: from Acceleration to Variational Inequalitiescited by 14
  22. 2014Distributed Robust Learningcited by 13
  23. 2020Distributed Momentum for Byzantine-resilient Learningcited by 10
  24. 2021Distributed Deep Learning Using Volunteer Computing-Like Paradigmcited by 9

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 4.

5 papers.

  1. 2025Prediction-Powered Semi-Supervised Learning with Online Power Tuningcited by 5
  2. 2025Bant: Byzantine Antidote via Trial Function and Trust Scorescited by 3
  3. 2025Robust Federated Inferencecited by 2
  4. 2024Beyond adaptive gradient: Fast-Controlled Minibatch Algorithm for large-scale optimizationcited by 0
  5. 2025Beyond Communication Overhead: A Multilevel Monte Carlo Approach for Mitigating Compression Bias in Distributed Learningcited by 0