mlsanityData to 14 Oct 2026

Reconciling Communication Compression and Byzantine-Robustness in Distributed Learning

arXiv 2508.17129 ↗First posted 23 Aug 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Distributed learning enables scalable model training over decentralized data, but remains hindered by Byzantine faults and high communication costs. While both challenges have been studied extensively in isolation, their interplay has received limited attention. Prior work has shown that naively combining communication compression with Byzantine-robust aggregation can severely weaken resilience to faulty nodes. The current state-of-the-art, Byz-DASHA-PAGE, leverages a momentum-based variance reduction scheme to counteract the negative effect of compression noise on Byzantine robustness. In this work, we introduce RoSDHB, a new algorithm that integrates classical Polyak momentum with a coordinated compression strategy. Theoretically, RoSDHB matches the convergence guarantees of Byz-DASHA-PAGE under the standard (G,B)-gradient dissimilarity model, while relying on milder assumptions and requiring less memory and communication per client. Empirically, RoSDHB demonstrates stronger robustness while achieving substantial communication savings compared to Byz-DASHA-PAGE.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 23 Aug 2025.

Builds on

25 of 48 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  4. 2018Byzantine-Robust Distributed Learning: Towards Optimal Statistical Ratescited by 711
  5. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
  6. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 571
  7. 2020Federated Learning with Matched Averagingcited by 564
  8. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 419
  9. 2018Sparsified SGD with Memorycited by 376
  10. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 356
  11. 2018The Hidden Vulnerability of Distributed Learning in Byzantiumcited by 329
  12. 2019Robust Aggregation for Federated Learningcited by 317
13 more
  1. 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 306
  2. 2019A Little Is Enough: Circumventing Defenses For Distributed Learningcited by 288
  3. 2019Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulationcited by 141
  4. 2020On Biased Compression for Distributed Learningcited by 140
  5. 2020Learning from History for Byzantine Robust Optimizationcited by 137
  6. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 133
  7. 2020Byzantine-Robust Learning on Heterogeneous Datasets via Bucketingcited by 128
  8. 2023Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneitycited by 65
  9. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  10. 2021Permutation Compressors for Provably Faster Distributed Nonconvex Optimizationcited by 32
  11. 2023Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneitycited by 26
  12. 2019Communication-Efficient and Byzantine-Robust Distributed Learning with Error Feedbackcited by 15
  13. 2023Communication Compression for Byzantine Robust Learning: New Efficient Algorithms and Improved Ratescited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Boundscited by 0