mlsanityData to 7 Oct 2026

cpSGD: Communication-efficient and differentially-private distributed SGD

arXiv 1805.10559 ↗First posted 27 May 2018stat.ML cs.CR cs.LGcited by 157

Not on the map. Why?

Abstract (arXiv)

Distributed stochastic gradient descent is an important subroutine in distributed learning. A setting of particular interest is when the clients are mobile devices, where two important concerns are communication efficiency and the privacy of the clients. Several recent works have focused on reducing the communication cost or introducing privacy guarantees, but none of the proposed communication efficient methods are known to be privacy preserving and none of the known privacy mechanisms are known to be communication efficient. To this end, we study algorithms that achieve both communication efficiency and differential privacy. For d variables and n ≈ d clients, the proposed method uses O(log log(nd)) bits of communication per client per coordinate and ensures constant privacy. We also extend and improve previous analysis of the Binomial mechanism showing that it achieves nearly the same utility as the Gaussian mechanism, while requiring fewer representation bits, which can be of independent interest.

Key facts. Cited by 157 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 41; 2022: 38; 2023: 34; 2024: 25; 2025: 14; 2026: 5.

157 papers.

Sort:

By area:

  1. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  2. 2021A Field Guide to Federated Optimizationcited by 312
  3. 2021Personalized Federated Learning using Hypernetworkscited by 210
  4. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  5. 2021CRFL: Certifiably Robust Federated Learning against Backdoor Attackscited by 91
  6. 2021Large Scale Private Learning via Low-rank Reparametrizationcited by 87
  7. 2021Personalized Federated Learning with Gaussian Processescited by 76
  8. 20211-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speedcited by 74
  9. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
  10. 2022FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learningcited by 65
  11. 2021Improved Rates for Differentially Private Stochastic Convex Optimization with Heavy-Tailed Datacited by 47
  12. 2022The Fundamental Price of Secure Aggregation in Differentially Private Federated Learningcited by 39
38 more
  1. 2022Gradient Obfuscation Gives a False Sense of Security in Federated Learningcited by 37
  2. 2022Optimal Algorithms for Mean Estimation under Local Differential Privacycited by 36
  3. 2022Differentially Private Federated Learning with Local Regularization and Sparsificationcited by 35
  4. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  5. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  6. 2023FedNoRo: Towards Noise-Robust Federated Learning by Addressing Class Imbalance and Label Noise Heterogeneitycited by 33
  7. 2021FedJAX: Federated learning simulation with JAXcited by 32
  8. 2022Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacycited by 32
  9. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31
  10. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  11. 2024Fine-Tuning Large Language Models with User-Level Differential Privacycited by 28
  12. 2021Lossless Compression of Efficient Private Local Randomizerscited by 27
  13. 2021An Efficient DP-SGD Mechanism for Large Scale NLP Modelscited by 27
  14. 2021Personalized Federated Learning with Adaptive Batchnorm for Healthcarecited by 26
  15. 2024Decentralized Federated Learning: A Survey on Security and Privacycited by 25
  16. 20211-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speedcited by 24
  17. 2023On the Privacy-Robustness-Utility Trilemma in Distributed Learningcited by 24
  18. 2023A Survey of Trustworthy Federated Learning with Perspectives on Security, Robustness, and Privacycited by 24
  19. 2021Privacy Amplification for Federated Learning via User Sampling and Wireless Aggregationcited by 22
  20. 2021Optimal Compression of Locally Differentially Private Mechanismscited by 22
  21. 2023Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimationcited by 22
  22. 2021Preserved central model for faster bidirectional compression in distributed settingscited by 21
  23. 2021Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFTcited by 20
  24. 2022On Distributed Adaptive Optimization with Gradient Compressioncited by 19
  25. 2022How Much Privacy Does Federated Learning with Secure Aggregation Guarantee?cited by 19
  26. 2021Wide Network Learning with Differential Privacycited by 17
  27. 2021Distributed Learning Systems with First-order Methodscited by 17
  28. 2022Federated Learning with Heterogeneous Architectures using Graph HyperNetworkscited by 17
  29. 2023z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learningcited by 17
  30. 2024The Privacy Power of Correlated Noise in Decentralized Learningcited by 17
  31. 2023Randomized Quantization is All You Need for Differential Privacy in Federated Learningcited by 16
  32. 2021Differential Privacy Meets Federated Learning under Communication Constraintscited by 15
  33. 2021FLoRA: Single-shot Hyper-parameter Optimization for Federated Learningcited by 14
  34. 2022Combined Federated and Split Learning in Edge Computing for Ubiquitous Intelligence in Internet of Things: State of the Art and Future Directionscited by 14
  35. 2023Fast Optimal Locally Private Mean Estimation via Random Projectionscited by 14
  36. 2023Securing Distributed SGD against Gradient Leakage Threatscited by 13
  37. 2023Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learningcited by 13
  38. 2022Bridging Differential Privacy and Byzantine-Robustness via Model Aggregationcited by 12

50 most cited of 157 All 157 (JSON)