mlsanityData to 7 Oct 2026

Communication-Computation Efficient Gradient Coding

arXiv 1802.03475 ↗First posted 9 Feb 2018stat.ML cs.DC cs.ITcited by 26

Not on the map. Why?

Abstract (arXiv)

This paper develops coding techniques to reduce the running time of distributed learning tasks. It characterizes the fundamental tradeoff to compute gradients (and more generally vector summations) in terms of three parameters: computation load, straggler tolerance and communication cost. It further gives an explicit coding scheme that achieves the optimal tradeoff based on recursive polynomial constructions, coding both across data subsets and vector components. As a result, the proposed scheme allows to minimize the running time for gradient computations. Implementations are made on Amazon EC2 clusters using Python with mpi4py package. Results show that the proposed scheme maintains the same generalization error while reducing the running time by 32% compared to uncoded schemes and 23% compared to prior coded schemes focusing only on stragglers (Tandon et al., ICML 2017).

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 7; 2023: 4; 2024: 2; 2025: 2; 2026: 2.

26 papers.

Sort:

  1. 20211-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speedcited by 74
  2. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  3. 20211-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speedcited by 24
  4. 2021Distributed Learning Systems with First-order Methodscited by 17
  5. 2023Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oraclescited by 16
  6. 2022Efficient Convex Optimization Requires Superlinear Memorycited by 14
  7. 2021Joint Optimization of Communications and Federated Learning Over the Aircited by 13
  8. 2021Gradient Coding with Dynamic Clustering for Straggler-Tolerant Distributed Learningcited by 8
  9. 2025Fundamental Limits of Hierarchical Secure Aggregation with Cyclic User Associationcited by 7
  10. 2021ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systemscited by 5
  11. 2021Optimal Communication-Computation Trade-Off in Heterogeneous Gradient Codingcited by 2
  12. 2022Adaptive Stochastic Gradient Descent for Fast and Communication-Efficient Distributed Learningcited by 2
14 more
  1. 2021Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossipingcited by 1
  2. 2022Lightweight Projective Derivative Codes for Compressed Asynchronous Gradient Descentcited by 1
  3. 2022Optimization-based Block Coordinate Gradient Coding for Mitigating Partial Stragglers in Distributed Learningcited by 1
  4. 2022Sequential Gradient Coding For Straggler Mitigationcited by 1
  5. 2021Solon: Communication-efficient Byzantine-resilient Distributed Training via Redundant Gradientscited by 0
  6. 2022Adaptive Worker Grouping For Communication-Efficient and Straggler-Tolerant Distributed SGDcited by 0
  7. 2022Cost-Efficient Distributed Learning via Combinatorial Multi-Armed Banditscited by 0
  8. 2023Fast and Straggler-Tolerant Distributed SGD with Reduced Computation Loadcited by 0
  9. 2023Iterative Sketching for Secure Coded Regressioncited by 0
  10. 2024Approximate Gradient Coding for Privacy-Flexible Federated Learning with Non-IID Datacited by 0
  11. 2024Design and Optimization of Hierarchical Gradient Coding for Distributed Learning at Edge Devicescited by 0
  12. 2025Approximate Gradient Coding for Distributed Learning with Heterogeneous Stragglerscited by 0
  13. 2026Coding-Enforced Resilient and Secure Aggregation for Hierarchical Federated Learningcited by 0
  14. 2026Pipelined Gradient Codingcited by 0