mlsanityData to 7 Oct 2026

Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting

arXiv 2302.06079 ↗First posted 13 Feb 2023cs.LG cs.AI cs.CRcited by 13

Not on the map. Why?

Abstract (arXiv)

Federated learning has exhibited vulnerabilities to Byzantine attacks, where the Byzantine attackers can send arbitrary gradients to a central server to destroy the convergence and performance of the global model. A wealth of robust AGgregation Rules (AGRs) have been proposed to defend against Byzantine attacks. However, Byzantine clients can still circumvent robust AGRs when data is non-Identically and Independently Distributed (non-IID). In this paper, we first reveal the root causes of performance degradation of current robust AGRs in non-IID settings: the curse of dimensionality and gradient heterogeneity. In order to address this issue, we propose GAS, a shorten approach that can successfully adapt existing robust AGRs to non-IID settings. We also provide a detailed convergence analysis when the existing robust AGRs are combined with GAS. Experiments on various real-world datasets verify the efficacy of our proposed GAS. The implementation code is provided in https://github.com/YuchenLiu-a/byzantine-gas.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

33 of 45 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  6. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  7. 2018LEAF: A Benchmark for Federated Settingscited by 780
  8. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  9. 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 566
  10. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  11. 2019Bayesian Nonparametric Federated Learning of Neural Networkscited by 385
  12. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 324
21 more
  1. 2019Robust Aggregation for Federated Learningcited by 314
  2. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
  3. 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
  4. 2020Threats to Federated Learning: A Surveycited by 171
  5. 2020Privacy and Robustness in Federated Learning: Attacks and Defensescited by 163
  6. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  7. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 136
  8. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 125
  9. 2021DENSE: Data-Free One-Shot Federated Learningcited by 100
  10. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 88
  11. 2017Mean Field Residual Networks: On the Edge of Chaoscited by 83
  12. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  13. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 56
  14. 2020Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)cited by 54
  15. 2020Byzantine-Resilient High-Dimensional Federated Learningcited by 27
  16. 2021Robust Training in High Dimensions via Block Coordinate Geometric Median Descentcited by 20
  17. 2022CalFAT: Calibrated Federated Adversarial Training with Label Skewnesscited by 20
  18. 2022Shielding Federated Learning: Robust Aggregation with Adaptive Client Selectioncited by 18
  19. 2022Secure Distributed Optimization Under Gradient Attackscited by 9
  20. 2020Robust Federated Recommendation Systemcited by 8
  21. 2020Byzantine-Robust Variance-Reduced Federated Learning over Distributed Non-i.i.d. Datacited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 2; 2026: 5.

13 papers.

Sort:

  1. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  2. 2025Do We Really Need to Design New Byzantine-robust Aggregation Rules?cited by 13
  3. 2024Certifiably Byzantine-Robust Federated Conformal Predictioncited by 8
  4. 2024Review of Mathematical Optimization in Federated Learningcited by 3
  5. 2024Understanding Byzantine Robustness in Federated Learning with A Black-box Servercited by 1
  6. 2024Aggressive or Imperceptible, or Both: Network Pruning Assisted Hybrid Byzantines in Federated Learningcited by 0
  7. 2024FedCAP: Robust Federated Learning via Customized Aggregation and Personalizationcited by 0
  8. 2025Approaching the Harm of Gradient Attacks While Only Flipping Labelscited by 0
  9. 2026FedAgain: A Trust-Based and Robust Federated Learning Strategy for an Automated Kidney Stone Identification in Ureteroscopycited by 0
  10. 2026AdaBFL: Multi-Layer Defensive Adaptive Aggregation for Bzantine-Robust Federated Learningcited by 0
  11. 2026mathsfVISTA: Decentralized Machine Learning in Adversary Dominated Environmentscited by 0
  12. 2026A Wolf in Sheep's Clothing: Targeted Routing Hijacking in Federated RAGcited by 0
1 more
  1. 2026On the Gradient Heterogeneity Dynamics of Adversarially Robust Federated Regressioncited by 0