mlsanityData to 7 Oct 2026

Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?

arXiv 2102.08166 ↗First posted 16 Feb 2021cs.LG cs.CR cs.DCcited by 17

Not on the map. Why?

Abstract (arXiv)

This paper addresses the problem of combining Byzantine resilience with privacy in machine learning (ML). Specifically, we study if a distributed implementation of the renowned Stochastic Gradient Descent (SGD) learning algorithm is feasible with both differential privacy (DP) and (α,f)-Byzantine resilience. To the best of our knowledge, this is the first work to tackle this problem from a theoretical point of view. A key finding of our analyses is that the classical approaches to these two (seemingly) orthogonal issues are incompatible. More precisely, we show that a direct composition of these techniques makes the guarantees of the resulting SGD algorithm depend unfavourably upon the number of parameters of the ML model, making the training of large models practically infeasible. We validate our theoretical results through numerical experiments on publicly-available datasets; showing that it is impractical to ensure DP and Byzantine resilience simultaneously.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 23 of its 45 references that are in the map. First posted on arXiv 16 Feb 2021.

Builds on

23 of 45 references.

References · 23

  1. 2016Wide Residual Networkscited by 3,057
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  5. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  8. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 542
  9. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  10. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  11. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 335
  12. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 324
11 more
  1. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
  2. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 163
  3. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 136
  4. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 104
  5. 2018Generalized Byzantine-tolerant SGDcited by 94
  6. 2020Byzantine-Resilient Secure Federated Learningcited by 93
  7. 2018DRACO: Byzantine-resilient Distributed Training via Redundant Gradientscited by 74
  8. 2020Secure Byzantine-Robust Machine Learningcited by 18
  9. 2018Phocas: dimensional Byzantine-resilient stochastic gradient descentcited by 13
  10. 2020Differentially Private Stochastic Coordinate Descentcited by 8
  11. 2020MYSTIKO : : Cloud-Mediated, Private, Federated Gradient Descentcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 1; 2023: 7; 2024: 3; 2025: 2; 2026: 1.

17 papers.

Sort:

By area:

  1. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  2. 2023Practical Differentially Private and Byzantine-resilient Federated Learningcited by 18
  3. 2023Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defensecited by 13
  4. 2022Bridging Differential Privacy and Byzantine-Robustness via Model Aggregationcited by 12
  5. 2024Adaptive Gradient Clipping for Robust Federated Learningcited by 10
  6. 2023On the Tradeoff between Privacy Preservation and Byzantine-Robustness in Decentralized Learningcited by 8
  7. 2023Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacycited by 8
  8. 2025Towards Trustworthy Federated Learning with Untrusted Participantscited by 8
  9. 2021Combining Differential Privacy and Byzantine Resilience in Distributed SGDcited by 3
  10. 2021PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacycited by 3
  11. 2024TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Datacited by 2
  12. 2021The More, the Better? A Study on Collaborative Machine Learning for DGA Detectioncited by 1
5 more
  1. 2023Balancing Privacy, Robustness, and Efficiency in Machine Learningcited by 1
  2. 2024Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixingcited by 1
  3. 2023SABLE: Secure And Byzantine robust LEarningcited by 0
  4. 2025Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learningcited by 0
  5. 2026Robust Federated Learning via Byzantine Filtering over Encrypted Updatescited by 0