mlsanityData to 7 Oct 2026

Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers

arXiv 2403.14716 ↗First posted 19 Mar 2024cs.LG cs.DCcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper considers the problem of distributed learning (DL) in the presence of stragglers. For this problem, DL methods based on gradient coding have been widely investigated, which redundantly distribute the training data to the workers to guarantee convergence when some workers are stragglers. However, these methods require the workers to transmit real-valued vectors during the process of learning, which induces very high communication burden. To overcome this drawback, we propose a novel DL method based on 1-bit gradient coding (1-bit GCDL), where 1-bit data encoded from the locally computed gradients are transmitted by the workers to reduce the communication overhead. We theoretically provide the convergence guarantees of the proposed method for both the convex loss functions and nonconvex loss functions. It is shown empirically that 1-bit GC-DL outperforms the baseline methods, which attains better learning performance under the same communication overhead.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 29 of its 53 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

29 of 53 references.

References · 29

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  3. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  4. 2019Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communicationcited by 218
  5. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  6. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  7. 2018Communication Compression for Decentralized Trainingcited by 87
  8. 2015Speeding Up Distributed Machine Learning Using Codescited by 76
  9. 2015Perturbed Iterate Analysis for Asynchronous Stochastic Optimizationcited by 67
  10. 2020Ternary Compression for Communication-Efficient Federated Learningcited by 57
  11. 2018Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communicationcited by 51
  12. 2019Stochastic Sign Descent Methods: New Algorithms and Better Theorycited by 49
17 more
  1. 2019Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradientscited by 32
  2. 2019High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learningcited by 32
  3. 2019ErasureHead: Distributed Gradient Descent without Delays Using Approximate Gradient Codingcited by 22
  4. 2019Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learningcited by 17
  5. 2019On Maintaining Linear Convergence of Distributed Learning and Optimization under Limited Communicationcited by 15
  6. 2020Approximate Gradient Coding with Optimal Decodingcited by 14
  7. 2019Gradient Coding with Clustering and Multi-message Communicationcited by 12
  8. 2021BROADCAST: Reducing Both Stochastic and Compression Noise to Robustify Communication-Efficient Federated Learningcited by 12
  9. 2019LAGC: Lazily Aggregated Gradient Coding for Straggler-Tolerant and Communication-Efficient Distributed Learningcited by 9
  10. 2021Distributed Online Learning with Multiple Kernelscited by 9
  11. 2022Distributed Learning With Sparsified Gradient Differencescited by 9
  12. 2021Gradient Coding with Dynamic Clustering for Straggler-Tolerant Distributed Learningcited by 8
  13. 2021In-network Computation for Large-scale Federated Learning over Wireless Edge Networkscited by 7
  14. 2020Distributed Learning with Dependent Samplescited by 5
  15. 2020Wireless Distributed Edge Learning: How Many Edge Devices Do We Need?cited by 3
  16. 2021START: Straggler Prediction and Mitigation for Cloud Computing Environments using Encoder LSTM Networkscited by 3
  17. 2021Dynamic Network-Assisted D2D-Aided Coded Distributed Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Adaptive Coded Federated Learning: Privacy Preservation and Straggler Mitigationcited by 1
  2. 2025Coded Robust Aggregation for Distributed Learning under Byzantine Attackscited by 0
  3. 2025Approximate Gradient Coding for Distributed Learning with Heterogeneous Stragglerscited by 0
  4. 2026A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraintscited by 0