mlsanityData to 7 Oct 2026

Robust Distributed Learning Against Both Distributional Shifts and Byzantine Attacks

arXiv 2210.16682 ↗First posted 29 Oct 2022cs.LG eess.SPcited by 0

Not on the map. Why?

Abstract (arXiv)

In distributed learning systems, robustness issues may arise from two sources. On one hand, due to distributional shifts between training data and test data, the trained model could exhibit poor out-of-sample performance. On the other hand, a portion of working nodes might be subject to byzantine attacks which could invalidate the learning result. Existing works mostly deal with these two issues separately. In this paper, we propose a new algorithm that equips distributed learning with robustness measures against both distributional shifts and byzantine attacks. Our algorithm is built on recent advances in distributionally robust optimization as well as norm-based screening (NBS), a robust aggregation scheme against byzantine attacks. We provide convergence proofs in three cases of the learning model being nonconvex, convex, and strongly convex for the proposed algorithm, shedding light on its convergence behaviors and endurability against byzantine attacks. In particular, we deduce that any algorithm employing NBS (including ours) cannot converge when the percentage of byzantine nodes is 1/3 or higher, instead of 1/2, which is the common belief in current literature. The experimental results demonstrate the effectiveness of our algorithm against both robustness issues. To the best of our knowledge, this is the first work to address distributional shifts and byzantine attacks simultaneously.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 72 references that are in the map. First posted on arXiv 29 Oct 2022.

Builds on

27 of 72 references.

References · 27

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  6. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  7. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  8. 2019Agnostic Federated Learningcited by 451
  9. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  10. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
  11. 2020FLTrust: Byzantine-robust Federated Learning via Trust Bootstrappingcited by 330
  12. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 324
15 more
  1. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  2. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
  3. 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 163
  4. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 148
  5. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 136
  6. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 125
  7. 2015Distributionally Robust Logistic Regressioncited by 124
  8. 2021Distributionally Robust Federated Averagingcited by 104
  9. 2018DRACO: Byzantine-resilient Distributed Training via Redundant Gradientscited by 74
  10. 2020Federated Learning Meets Multi-objective Optimizationcited by 50
  11. 2019DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregationcited by 44
  12. 2020A Reputation Mechanism Is All You Need: Collaborative Fairness and Adversarial Robustness in Federated Learningcited by 38
  13. 2020Distributed Newton Can Communicate Less and Resist Byzantine Workerscited by 20
  14. 2019Communication-Efficient and Byzantine-Robust Distributed Learning with Error Feedbackcited by 15
  15. 2020Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Datacited by 3

Built on by

Not cited yet.