mlsanityData to 14 Oct 2026

TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data

arXiv 2402.10816 ↗First posted 16 Feb 2024cs.LG cs.CR cs.DCcited by 2

Not on the map. Why?

Abstract (arXiv)

Distributed training of deep neural networks faces three critical challenges: privacy preservation, communication efficiency, and robustness to fault and adversarial behaviors. Although significant research efforts have been devoted to addressing these challenges independently, their synthesis remains less explored. In this paper, we propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the emerging f-differential privacy (DP) and the Byzantine resilience of the proposed algorithm. Particularly, in terms of privacy guarantees, compared to the existing sign-based approach StoSign, the proposed method improves the dimension dependence on the gradient size and enjoys privacy amplification by mini-batch sampling while ensuring a comparable convergence rate. We also prove that TernaryVote is robust when less than 50% of workers are blind attackers, which matches that of SIGNSGD with majority vote. Extensive experimental results validate the effectiveness of the proposed algorithm.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 43 of its 59 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

43 of 59 references.

References · 43

  1. 2019Advances and Open Problems in Federated Learningcited by 3,046
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2019Deep Leakage from Gradientscited by 1,021
  4. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  5. 2020The Future of Digital Health with Federated Learningcited by 633
  6. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  7. 2018Sparsified SGD with Memorycited by 376
  8. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  9. 2019Error Feedback Fixes SignSGD and other Gradient Compression Schemescited by 310
  10. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
  11. 2019Can You Really Backdoor Federated Learning?cited by 255
  12. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
31 more
  1. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
  2. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 158
  3. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 149
  4. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 141
  5. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  6. 2020The Discrete Gaussian for Differential Privacycited by 135
  7. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 128
  8. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 128
  9. 2018Generalized Byzantine-tolerant SGDcited by 95
  10. 2020Local and Central Differential Privacy for Robustness and Privacy in Federated Learningcited by 78
  11. 2021Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacycited by 69
  12. 2020Breaking the Communication-Privacy-Accuracy Trilemmacited by 68
  13. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 68
  14. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  15. 2019Stochastic Sign Descent Methods: New Algorithms and Better Theorycited by 49
  16. 2022Gradient Obfuscation Gives a False Sense of Security in Federated Learningcited by 39
  17. 2020Stochastic-Sign SGD for Federated Learning with Theoretical Guaranteescited by 35
  18. 2022Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guaranteescited by 30
  19. 2019Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithmscited by 29
  20. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  21. 2023Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimationcited by 22
  22. 2021Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?cited by 18
  23. 2022Privacy-Aware Compression for Federated Data Analysiscited by 17
  24. 2022Bridging Differential Privacy and Byzantine-Robustness via Model Aggregationcited by 13
  25. 2018signSGD with Majority Vote is Communication Efficient And Fault Tolerantcited by 11
  26. 2021Improving Differentially Private SGD via Randomly Sparsified Gradientscited by 8
  27. 2023Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacycited by 8
  28. 2022Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Designcited by 6
  29. 2016The Team Surviving Orienteers Problem: Routing Robots in Uncertain Environments with Survival Constraintscited by 4
  30. 2022Distributed Non-Convex Optimization with One-Bit Compressors on Heterogeneous Data: Efficient and Resilient Algorithmscited by 4
  31. 2023Breaking the Communication-Privacy-Accuracy Tradeoff with f-Differential Privacycited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learningcited by 0
  2. 2025Mobility-Assisted Decentralized Federated Learning: Convergence Analysis and A Data-Driven Approachcited by 0