mlsanityData to 14 Oct 2026

SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding

arXiv 2402.01340 ↗First posted 2 Feb 2024cs.LG cs.CR eess.SPcited by 1

Not on the map. Why?

Abstract (arXiv)

Distributed learning is an effective approach to accelerate model training using multiple workers. However, substantial communication delays emerge between workers and a parameter server due to massive costs associated with communicating gradients. SignSGD with majority voting (signSGD-MV) is a simple yet effective optimizer that reduces communication costs through one-bit quantization, yet the convergence rates considerably decrease as adversarial workers increase. In this paper, we show that the convergence rate is invariant as the number of adversarial workers increases, provided that the number of adversarial workers is smaller than that of benign workers. The key idea showing this counter-intuitive result is our novel signSGD with federated defense (signSGD-FD). Unlike the traditional approaches, signSGD-FD exploits the gradient information sent by adversarial workers with the proper weights, which are obtained through gradient sign decoding. Experimental results demonstrate signSGD-FD achieves superior convergence rates over traditional algorithms in various adversarial attack scenarios.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 50 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

29 of 50 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2023A Survey of Large Language Modelscited by 2,576
  4. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  5. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  6. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  7. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  8. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  9. 2019Robust Aggregation for Federated Learningcited by 317
  10. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
  11. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 309
  12. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
17 more
  1. 2017Sparse Communication for Distributed Gradient Descentcited by 249
  2. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 181
  3. 2020FetchSGD: Communication-Efficient Federated Learning with Sketchingcited by 179
  4. 2020Threats to Federated Learning: A Surveycited by 178
  5. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 141
  6. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  7. 2018Byzantine Stochastic Gradient Descentcited by 81
  8. 2020Fast-Convergent Federated Learning with Adaptive Weightingcited by 67
  9. 2023Revisiting Weighted Aggregation in Federated Learning with Neural Networkscited by 58
  10. 2019Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedbackcited by 50
  11. 2020Stochastic-Sign SGD for Federated Learning with Theoretical Guaranteescited by 35
  12. 2019Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithmscited by 29
  13. 2022Near-Optimal Sparse Allreduce for Distributed Deep Learningcited by 23
  14. 2014Error Rate Bounds and Iterative Weighted Majority Voting for Crowdsourcingcited by 21
  15. 2019Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attackscited by 20
  16. 2013Large Deviation Methods for Approximate Probabilistic Inferencecited by 12
  17. 2021A Worker-Task Specialization Model for Crowdsourcing: Efficient Inference and Fundamental Limitscited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Approximate Agreement Algorithms for Byzantine Collaborative Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024SignSGD with Federated Voting
  2. 2023Sparse-SignSGD with Majority Vote for Communication-Efficient Distributed Learning
  3. 2026Enhancing SignSGD: Small-Batch Convergence Analysis and a Hybrid Switching Strategy
  4. 2026Fast Adversarial Attacks with Gradient Prediction
  5. 2026SignMuon: Communication-Efficient Distributed Muon Optimization
  6. 2025FedBAP: Backdoor Defense via Benign Adversarial Perturbation in Federated Learning
  7. 2023FedGrad: Mitigating Backdoor Attacks in Federated Learning Through Local Ultimate Gradients Inspection
  8. 2024Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries