mlsanityData to 7 Oct 2026

Byzantine-Robust Decentralized Learning via ClippedGossip

arXiv 2202.01545 ↗First posted 3 Feb 2022cs.LG cs.DC math.OCcited by 29

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the challenging task of Byzantine-robust decentralized training on arbitrary communication graphs. Unlike federated learning where workers communicate through a server, workers in the decentralized environment can only talk to their neighbors, making it harder to reach consensus and benefit from collaborative training. To address these issues, we propose a ClippedGossip algorithm for Byzantine-robust consensus and optimization, which is the first to provably converge to a O(δmaxzeta2/γ2) neighborhood of the stationary point for non-convex objectives under standard assumptions. Finally, we demonstrate the encouraging empirical performance of ClippedGossip under a large number of attacks.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 47 of its 83 references that are in the map. First posted on arXiv 3 Feb 2022.

Builds on

47 of 83 references.

References · 47

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 698
  6. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  7. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 361
  8. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 324
  9. 2019Robust Aggregation for Federated Learningcited by 314
  10. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 285
  11. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 218
  12. 2018Stochastic Gradient Push for Distributed Deep Learningcited by 152
35 more
  1. 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 148
  2. 2018D2: Decentralized Training over Decentralized Datacited by 140
  3. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  4. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 136
  5. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 125
  6. 2019Decentralized Deep Learning with Arbitrary Communication Compressioncited by 109
  7. 2018Generalized Byzantine-tolerant SGDcited by 94
  8. 2018Byzantine Stochastic Gradient Descentcited by 81
  9. 2018DRACO: Byzantine-resilient Distributed Training via Redundant Gradientscited by 74
  10. 2021Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Datacited by 69
  11. 2020Byzantine-Resilient Non-Convex Stochastic Gradient Descentcited by 56
  12. 2021Exponential Graph is Provably Efficient for Decentralized Deep Trainingcited by 55
  13. 2020Collaborative Learning in the Jungle (Decentralized, Byzantine, Heterogeneous, Asynchronous and Nonconvex Learning)cited by 54
  14. 2019Zeno++: Robust Fully Asynchronous SGDcited by 53
  15. 2021Consensus Control for Decentralized Deep Learningcited by 49
  16. 2019BRIDGE: Byzantine-resilient Decentralized Gradient Descentcited by 45
  17. 2019DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregationcited by 44
  18. 2020A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!cited by 41
  19. 2021DecentLaM: Decentralized Momentum SGD for Large-batch Deep Trainingcited by 40
  20. 2021RelaySum for Decentralized Deep Learning on Heterogeneous Datacited by 39
  21. 2017ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learningcited by 36
  22. 2020PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learningcited by 35
  23. 2018Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learningcited by 32
  24. 2020Byzantine-resilient Decentralized Stochastic Gradient Descentcited by 28
  25. 2019Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attackscited by 20
  26. 2016Distributed Optimization Under Adversarial Nodescited by 19
  27. 2020Secure Byzantine-Robust Machine Learningcited by 18
  28. 2020Byzantine-Resilient SGD in High Dimensions on Heterogeneous Datacited by 17
  29. 2021BlueFog: Make Decentralized Algorithms Practical for Optimization and Deep Learningcited by 16
  30. 2021Secure Distributed Training at Scalecited by 11
  31. 2020Byzantine-Robust Decentralized Stochastic Optimization over Static and Time-Varying Networkscited by 10
  32. 2018Aggregation over Metric Spaces: Proposing and Voting in Elections, Budgeting, and Legislationcited by 7
  33. 2021Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancycited by 7
  34. 2019Building a Sybil-Resilient Digital Community Utilizing Trust-Graph Connectivitycited by 3
  35. 2018Communication Complexity of Byzantine Agreement, Revisitedcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 11; 2026: 11.

29 papers.

Sort:

  1. 2023Backdoor Attacks in Peer-to-Peer Federated Learningcited by 9
  2. 2024Unified Breakdown Analysis for Byzantine Robust Gossipcited by 8
  3. 2023Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differencescited by 5
  4. 2024Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Trainingcited by 3
  5. 2025Robust Distributed Estimation: Extending Gossip Algorithms to Ranking and Trimmed Meanscited by 3
  6. 2025Don't Reach for the Stars: Rethinking Topology for Resilient Federated Learningcited by 3
  7. 2025Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participationcited by 3
  8. 2023Byzantine-Robust Decentralized Stochastic Optimization with Stochastic Gradient Noise-Independent Learning Errorcited by 2
  9. 2025DICE: Data Influence Cascade in Decentralized Learningcited by 2
  10. 2024Byzantine-Robust Gossip: Insights from a Dual Approachcited by 1
  11. 2025Federated prediction for scalable and privacy-preserved knowledge-based planning in radiotherapycited by 1
  12. 2025Enhancing Parallelism in Decentralized Stochastic Convex Optimizationcited by 1
17 more
  1. 2025Screen Before You Fetch: Compressed Byzantine Screening for Decentralized Learning on the Edge-Cloud Continuumcited by 1
  2. 2024Impact of Network Topology on Byzantine Resilience in Decentralized Federated Learningcited by 0
  3. 2025GRANITE : a Byzantine-Resilient Dynamic Gossip Learning Frameworkcited by 0
  4. 2025ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learningcited by 0
  5. 2025Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learningcited by 0
  6. 2025Robust and Efficient Collaborative Learningcited by 0
  7. 2026Topology-Independent Robustness of the Weighted Mean under Label Poisoning Attacks in Heterogeneous Decentralized Learningcited by 0
  8. 2026Decentralized Ranking Aggregation via Gossip: Convergence and Robustnesscited by 0
  9. 2026Convergence of Byzantine-Resilient Gradient Tracking via Probabilistic Edge Dropoutcited by 0
  10. 2026Near-Optimal Decentralized Stochastic Convex Optimization over Networkscited by 0
  11. 2026Beyond Parameter Space: NTK-Guided Personalized Aggregation for Robust Federated Learningcited by 0
  12. 2026BackDFL: A Unified Benchmark For Backdoor Attacks and Defenses In Decentralized Federated Learningcited by 0
  13. 2026Model-Consistent Byzantine-Resilient Decentralized Federated Learning for Collaborative Missionscited by 0
  14. 2026Optimizing Byzantine Node Placement in Decentralized Federated Learningcited by 0
  15. 2026Robust Decentralized Federated Distillation via Multi-Modality Knowledge Collaborationcited by 0
  16. 2026Fully Byzantine-Resilient Multi-Agent Reinforcement Learningcited by 0
  17. 2026Robust Decentralized Fairness Auditingcited by 0