mlsanityData to 7 Oct 2026

Compression with Exact Error Distribution for Federated Learning

arXiv 2310.20682 ↗First posted 31 Oct 2023cs.LG cs.IT math.ITcited by 9

Not on the map. Why?

Abstract (arXiv)

Compression schemes have been extensively used in Federated Learning (FL) to reduce the communication cost of distributed learning. While most approaches rely on a bounded variance assumption of the noise produced by the compressor, this paper investigates the use of compression and aggregation schemes that produce a specific error distribution, e.g., Gaussian or Laplace, on the aggregated data. We present and analyze different aggregation schemes based on layered quantizers achieving exact error distribution. We provide different methods to leverage the proposed compression schemes to obtain compression-for-free in differential privacy applications. Our general compression methods can recover and improve standard FL schemes with Gaussian perturbations such as Langevin dynamics and randomized smoothing.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 37 of its 73 references that are in the map. First posted on arXiv 31 Oct 2023.

Builds on

37 of 73 references.

References · 37

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  5. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  8. 2017Renyi Differential Privacycited by 643
  9. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 528
  10. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  11. 2017TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learningcited by 253
  12. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 237
25 more
  1. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  2. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
  3. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 148
  4. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  5. 2020The Discrete Gaussian for Differential Privacycited by 134
  6. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  7. 2022Lossy Compression with Gaussian Diffusioncited by 87
  8. 2022An Introduction to Neural Data Compressioncited by 78
  9. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  10. 2020Universally Quantized Neural Compressioncited by 60
  11. 2018Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameterscited by 56
  12. 2018Optimal Algorithms for Non-Smooth Distributed Optimization in Networkscited by 41
  13. 2022The Fundamental Price of Secure Aggregation in Differentially Private Federated Learningcited by 39
  14. 2020Compressing Images by Encoding Their Latent Representations with Relative Entropy Codingcited by 36
  15. 2022Joint Privacy Enhancement and Quantization in Federated Learningcited by 34
  16. 2021QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learningcited by 26
  17. 2021Decentralized Federated Learning for UAV Networks: Architecture, Challenges, and Opportunitiescited by 22
  18. 2021Optimal Compression of Locally Differentially Private Mechanismscited by 22
  19. 2023Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimationcited by 22
  20. 2021Preserved central model for faster bidirectional compression in distributed settingscited by 21
  21. 2022Privacy-Aware Compression for Federated Data Analysiscited by 17
  22. 2022Communication-Efficient Robust Federated Learning with Noisy Labelscited by 11
  23. 2021A Graph Federated Architecture with Privacy Preserving Learningcited by 10
  24. 2023Communication-Efficient Laplace Mechanism for Differential Privacy via Random Quantizationcited by 8
  25. 2023Communication Efficient Private Federated Learning Using Ditheringcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 5.

9 papers.

  1. 2024Universal Exact Compression of Differentially Private Mechanismscited by 8
  2. 2024The Rate-Distortion-Perception Trade-off: The Role of Private Randomnesscited by 6
  3. 2024Distributed Event-Based Learning via ADMMcited by 3
  4. 2025Communication-Efficient and Privacy-Adaptable Mechanism for Federated Learningcited by 2
  5. 2026Communication-Efficient and Privacy-Adaptable Mechanism -- a Federated Learning Scheme with Convergence Analysiscited by 1
  6. 2026Deep Randomized Distributed Function Computation (DeepRDFC): Neural Distributed Channel Simulationcited by 1
  7. 2026Secure Rate-Distortion-Perception: A Randomized Distributed Function Computation Approach for Realismcited by 0
  8. 2026Scalable Discrete-to-Continuous Channel Simulation for Compression and Privacycited by 0
  9. 2026Privacy-Aligned Personalized Federated Learning with Compact Adaptation and Variable-Length Gaussian Communicationcited by 0