mlsanityData to 7 Oct 2026

CSER: Communication-efficient SGD with Error Reset

arXiv 2007.13221 ↗First posted 26 Jul 2020cs.LG cs.DC stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

The scalability of Distributed Stochastic Gradient Descent (SGD) is today limited by communication bottlenecks. We propose a novel SGD variant: Communication-efficient SGD with Error Reset, or CSER. The key idea in CSER is first a new technique called "error reset" that adapts arbitrary compressors for SGD, producing bifurcated local models with periodic reset of resulting local residual errors. Second we introduce partial synchronization for both the gradients and the models, leveraging advantages from them. We prove the convergence of CSER for smooth non-convex problems. Empirical results show that when combined with highly aggressive compressors, the CSER algorithms accelerate the distributed training by nearly 10x for CIFAR-100, and by 4.5x for ImageNet.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 4; 2023: 8; 2024: 4; 2025: 0; 2026: 1.

27 papers.

Sort:

  1. 2021A Field Guide to Federated Optimizationcited by 312
  2. 2021EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedbackcited by 131
  3. 20211-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speedcited by 74
  4. 2021Rethinking gradient sparsification as total error minimizationcited by 41
  5. 2022Lower Bounds and Nearly Optimal Algorithms in Distributed Learning with Communication Compressioncited by 33
  6. 2023Momentum Provably Improves Error Feedback!cited by 31
  7. 2022Fine-tuning Language Models over Slow Networks using Activation Compression with Guaranteescited by 29
  8. 2021Time-Correlated Sparsification for Communication-Efficient Federated Learningcited by 25
  9. 20211-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speedcited by 24
  10. 2022lo-fi: distributed fine-tuning without communicationcited by 19
  11. 2023Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compressioncited by 11
  12. 2023Efficient Asynchronous Federated Learning with Sparsification and Quantizationcited by 9
15 more
  1. 2021On Faster Convergence of Scaled Sign Gradient Descentcited by 6
  2. 2021EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedbackcited by 6
  3. 2021Distributed Online Learning for Joint Regret with Communication Constraintscited by 5
  4. 2021Compressed Communication for Distributed Training: Adaptive Methods and Systemcited by 5
  5. 2024Ordered Momentum for Asynchronous SGDcited by 5
  6. 2023Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication Compressioncited by 4
  7. 2023FedREP: A Byzantine-Robust, Communication-Efficient and Privacy-Preserving Framework for Federated Learningcited by 3
  8. 2024Communication-Efficient Distributed Learning with Local Immediate Error Compensationcited by 2
  9. 2022Communication-Efficient Adam-Type Algorithms for Distributed Data Miningcited by 1
  10. 2024Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysiscited by 1
  11. 2024Distributed Sign Momentum with Local Steps for Training Transformerscited by 1
  12. 2023Quantize Once, Train Fast: Allreduce-Compatible Compression with Provable Guaranteescited by 0
  13. 2023Contractive error feedback for gradient compressioncited by 0
  14. 2023RS-DGC: Exploring Neighborhood Statistics for Dynamic Gradient Compression on Remote Sensing Image Interpretationcited by 0
  15. 2026SA-PEF: Step-Ahead Partial Error Feedback for Efficient Federated Learningcited by 0