mlsanityData to 14 Oct 2026

Bridging Differential Privacy and Byzantine-Robustness via Model Aggregation

arXiv 2205.00107 ↗First posted 29 Apr 2022cs.LG cs.CR cs.DCcited by 13

Not on the map. Why?

Abstract (arXiv)

This paper aims at jointly addressing two seemly conflicting issues in federated learning: differential privacy (DP) and Byzantine-robustness, which are particularly challenging when the distributed data are non-i.i.d. (independent and identically distributed). The standard DP mechanisms add noise to the transmitted messages, and entangles with robust stochastic gradient aggregation to defend against Byzantine attacks. In this paper, we decouple the two issues via robust stochastic model aggregation, in the sense that our proposed DP mechanisms and the defense against Byzantine attacks have separated influence on the learning performance. Leveraging robust stochastic model aggregation, at each iteration, each worker calculates the difference between the local model and the global one, followed by sending the element-wise signs to the master node, which enables robustness to Byzantine attacks. Further, we design two DP mechanisms to perturb the uploaded signs for the purpose of privacy preservation, and prove that they are (ε,0)-DP by exploiting the properties of noise distributions. With the tools of Moreau envelop and proximal point projection, we establish the convergence of the proposed algorithm when the cost function is nonconvex. We analyze the trade-off between privacy preservation and learning performance, and show that the influence of our proposed DP mechanisms is decoupled with that of robust stochastic model aggregation. Numerical experiments demonstrate the effectiveness of the proposed algorithm.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 29 Apr 2022.

Builds on

19 of 33 references.

References · 19

  1. 2019Advances and Open Problems in Federated Learningcited by 3,046
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  4. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  5. 2017Renyi Differential Privacycited by 650
  6. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  7. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  8. 2013The Composition Theorem for Differential Privacycited by 297
  9. 2018Stochastic model-based minimization of weakly convex functionscited by 159
  10. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 158
  11. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 149
  12. 2020Learning from History for Byzantine Robust Optimizationcited by 137
7 more
  1. 2019Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attackscited by 79
  2. 2017Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descentcited by 59
  3. 2020Stochastic-Sign SGD for Federated Learning with Theoretical Guaranteescited by 35
  4. 2017Proximally Guided Stochastic Subgradient Method for Nonsmooth, Nonconvex Problemscited by 30
  5. 2021Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?cited by 18
  6. 2019Byzantine Resilient Non-Convex SVRG with Distributed Batch Gradient Computationscited by 6
  7. 2021Combining Differential Privacy and Byzantine Resilience in Distributed SGDcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 2; 2025: 3; 2026: 2.

13 papers.

Sort:

  1. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  2. 2023Practical Differentially Private and Byzantine-resilient Federated Learningcited by 18
  3. 2024Adaptive Gradient Clipping for Robust Federated Learningcited by 11
  4. 2023On the Tradeoff between Privacy Preservation and Byzantine-Robustness in Decentralized Learningcited by 8
  5. 2023Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacycited by 8
  6. 2025Towards Trustworthy Federated Learning with Untrusted Participantscited by 8
  7. 2025Cellular Traffic Prediction via Byzantine-robust Asynchronous Federated Learningcited by 4
  8. 2024TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Datacited by 2
  9. 2023Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysiscited by 1
  10. 2026Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptionscited by 1
  11. 2023SABLE: Secure And Byzantine robust LEarningcited by 0
  12. 2025Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learningcited by 0
1 more
  1. 2026AsynDBT: Asynchronous Distributed Bilevel Tuning for efficient In-Context Learning with Large Language Modelscited by 0