mlsanityData to 14 Oct 2026

DP-REC: Private & Communication-Efficient Federated Learning

arXiv 2111.05454 ↗First posted 9 Nov 2021cs.LG cs.CR stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Privacy and communication efficiency are important challenges in federated training of neural networks, and combining them is still an open problem. In this work, we develop a method that unifies highly compressed communication and differential privacy (DP). We introduce a compression technique based on Relative Entropy Coding (REC) to the federated setting. With a minor modification to REC, we obtain a provably differentially private learning algorithm, DP-REC, and show how to compute its privacy guarantees. Our experiments demonstrate that DP-REC drastically reduces communication costs while providing privacy guarantees comparable to the state-of-the-art.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 27 of its 46 references that are in the map. First posted on arXiv 9 Nov 2021.

Builds on

27 of 46 references.

References · 27

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2019Advances and Open Problems in Federated Learningcited by 3,046
  6. 2016Deep Learning with Differential Privacycited by 2,728
  7. 2020Adaptive Federated Optimizationcited by 1,046
  8. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  9. 2018LEAF: A Benchmark for Federated Settingscited by 792
  10. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  11. 2017Renyi Differential Privacycited by 650
  12. 2020Scaling Laws for Autoregressive Generative Modelingcited by 522
15 more
  1. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  2. 2021A Field Guide to Federated Optimizationcited by 315
  3. 2019Differentially Private Learning with Adaptive Clippingcited by 220
  4. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 210
  5. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 204
  6. 2020Characterising Bias in Compressed Modelscited by 167
  7. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 158
  8. 2020The Discrete Gaussian for Differential Privacycited by 135
  9. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 128
  10. 2020Breaking the Communication-Privacy-Accuracy Trilemmacited by 68
  11. 2018Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameterscited by 58
  12. 2020Compressing Images by Encoding Their Latent Representations with Relative Entropy Codingcited by 39
  13. 2020Understanding Unintended Memorization in Federated Learningcited by 36
  14. 2020Shuffled Model of Federated Learning: Privacy, Communication and Accuracy Trade-offscited by 15
  15. 2018Differentially-Private "Draw and Discard" Machine Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 2; 2024: 2; 2025: 0; 2026: 1.

10 papers.

  1. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  2. 2022Accelerated Federated Learning with Decoupled Adaptive Optimizationcited by 23
  3. 2022Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devicescited by 12
  4. 2023Communication Efficient Private Federated Learning Using Ditheringcited by 9
  5. 2024Universal Exact Compression of Differentially Private Mechanismscited by 8
  6. 2022Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Designcited by 6
  7. 2023Adaptive Compression in Federated Learning via Side Informationcited by 6
  8. 2022Reconciling Security and Communication Efficiency in Federated Learningcited by 4
  9. 2026Scalable Differentially Private Data Compression via Diffusion and Stochastic Codescited by 3
  10. 2024On Using Secure Aggregation in Differentially Private Federated Learning with Multiple Local Stepscited by 1