mlsanityData to 14 Oct 2026

Wyner-Ziv Gradient Compression for Federated Learning

arXiv 2111.08277 ↗First posted 16 Nov 2021cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Due to limited communication resources at the client and a massive number of model parameters, large-scale distributed learning tasks suffer from communication bottleneck. Gradient compression is an effective method to reduce communication load by transmitting compressed gradients. Motivated by the fact that in the scenario of stochastic gradients descent, gradients between adjacent rounds may have a high correlation since they wish to learn the same model, this paper proposes a practical gradient compression scheme for federated learning, which uses historical gradients to compress gradients and is based on Wyner-Ziv coding but without any probabilistic assumption. We also implement our gradient quantization method on the real dataset, and the performance of our method is better than the previous schemes.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 16 Nov 2021.

Builds on

16 of 29 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019Advances and Open Problems in Federated Learningcited by 3,046
  3. 2021Convex optimizationcited by 2,515
  4. 2018Sparsified SGD with Memorycited by 376
  5. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 256
  6. 2018The Convergence of Sparsified Gradient Methodscited by 229
  7. 2017Gradient Sparsification for Communication-Efficient Distributed Optimizationcited by 201
  8. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 160
  9. 2016QSGD: Communication-Efficient SGD via Gradient Quantization and Encodingcited by 147
  10. 2016Distributed Mean Estimation with Limited Communicationcited by 136
  11. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 136
  12. 2020Breaking the Communication-Privacy-Accuracy Trilemmacited by 68
4 more
  1. 2020Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressorcited by 45
  2. 2016Randomized Distributed Mean Estimation: Accuracy vs Communicationcited by 39
  3. 2020Optimal Gradient Compression for Distributed and Federated Learningcited by 39
  4. 2019RATQ: A Universal Fixed-Length Quantizer for Stochastic Optimizationcited by 23

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1.

3 papers.

  1. 2022A Survey on Over-the-Air Computationcited by 21
  2. 2022Over-the-Air Computation Based on Balanced Number Systems for Federated Edge Learningcited by 10
  3. 2023GQFedWAvg: Optimization-Based Quantized Federated Learning in General Edge Computing Systemscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2021Communication-Efficient Federated Learning via Predictive Coding
  2. 2021Communication-Efficient Federated Learning via Quantized Compressed Sensing
  3. 2024Rate-Constrained Quantization for Communication-Efficient Federated Learning
  4. 2025An Efficient Gradient-Aware Error-Bounded Lossy Compressor for Federated Learning
  5. 2024FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
  6. 2022Sparse Random Networks for Communication-Efficient Federated Learning
  7. 2024Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression
  8. 2023FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications