mlsanityData to 7 Oct 2026

Kimad: Adaptive Gradient Compression with Bandwidth Awareness

arXiv 2312.08053 ↗First posted 13 Dec 2023cs.LG cs.DC cs.ITcited by 2

Not on the map. Why?

Abstract (arXiv)

In distributed training, communication often emerges as a bottleneck. In response, we introduce Kimad, a solution that offers adaptive gradient compression. By consistently monitoring bandwidth, Kimad refines compression ratios to match specific neural network layer requirements. Our exhaustive tests and proofs confirm Kimad's outstanding performance, establishing it as a benchmark in adaptive compression for distributed deep learning.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 20 of its 38 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

20 of 38 references.

References · 20

  1. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  2. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  3. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  4. 2018Sparsified SGD with Memorycited by 373
  5. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  6. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
  7. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
  8. 2018The Convergence of Sparsified Gradient Methodscited by 228
  9. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 135
  10. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  11. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
  12. 2019Distributed Learning with Compressed Gradient Differencescited by 110
8 more
  1. 2019First Analysis of Local GD on Heterogeneous Datacited by 80
  2. 2016Randomized Distributed Mean Estimation: Accuracy vs Communicationcited by 39
  3. 20223PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregationcited by 27
  4. 2022Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezingcited by 22
  5. 2018Parallax: Sparsity-aware Data Parallel Training of Deep Neural Networkscited by 19
  6. 2020Accordion: Adaptive Gradient Communication via Critical Learning Regime Identificationcited by 19
  7. 2021CGX: Adaptive System Support for Communication-Efficient Deep Learningcited by 4
  8. 2022L-GreCo: Layerwise-Adaptive Gradient Compression for Efficient and Accurate Deep Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Layer-wise Quantization for Quantized Optimistic Dual Averagingcited by 1
  2. 2026CIERA: Cross-Iteration Exponent Reuse for Lossless Allgather in Sharded MoE Trainingcited by 0