mlsanityData to 7 Oct 2026

The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation

arXiv 2102.06387 ↗First posted 12 Feb 2021cs.LG cs.DS stat.MLcited by 127

Not on the map. Why?

Abstract (arXiv)

We consider training models on private data that are distributed across user devices. To ensure privacy, we add on-device noise and use secure aggregation so that only the noisy sum is revealed to the server. We present a comprehensive end-to-end system, which appropriately discretizes the data and adds discrete Gaussian noise before performing secure aggregation. We provide a novel privacy analysis for sums of discrete Gaussians and carefully analyze the effects of data quantization and modular summation arithmetic. Our theoretical guarantees highlight the complex tension between communication, privacy, and accuracy. Our extensive experimental results demonstrate that our solution is essentially able to match the accuracy to central differential privacy with less than 16 bits of precision per value.

Key facts. Cited by 127 papers in the map (data through 7 Oct 2026). Builds on 45 of its 75 references that are in the map. First posted on arXiv 12 Feb 2021.

Builds on

45 of 75 references.

References · 45

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Advances and Open Problems in Federated Learningcited by 3,021
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Adaptive Federated Optimizationcited by 1,036
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  8. 2019Towards Federated Learning at Scale: System Designcited by 826
  9. 2018LEAF: A Benchmark for Federated Settingscited by 780
  10. 2017Renyi Differential Privacycited by 643
  11. 2018Federated Learning for Mobile Keyboard Predictioncited by 565
  12. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 509
33 more
  1. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  2. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  3. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  4. 2019Differentially Private Learning with Adaptive Clippingcited by 218
  5. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 216
  6. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  7. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  8. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  9. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  10. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 163
  11. 2018cpSGD: Communication-efficient and differentially-private distributed SGDcited by 157
  12. 2016Concentrated Differential Privacycited by 139
  13. 2016Distributed Mean Estimation with Limited Communicationcited by 134
  14. 2020The Discrete Gaussian for Differential Privacycited by 134
  15. 2018Distributed Differential Privacy via Shufflingcited by 115
  16. 2019Federated Learning for Emoji Prediction in a Mobile Keyboardcited by 115
  17. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 104
  18. 2019Hypothesis Testing Interpretations and Renyi Differential Privacycited by 84
  19. 2020Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithmscited by 78
  20. 2019Overlearning Reveals Sensitive Attributescited by 74
  21. 2019The Privacy Blanket of the Shuffle Modelcited by 73
  22. 2016Discrete Distribution Estimation under Local Privacycited by 58
  23. 2017Prochlo: Strong Privacy for Analytics in the Crowdcited by 51
  24. 2019Average-Case Averages: Private Algorithms for Smooth Sensitivity and Mean Estimationcited by 51
  25. 2020Private Summation in the Multi-Message Shuffle Modelcited by 34
  26. 2019Federated Learning with Autotuned Communication-Efficient Secure Aggregationcited by 23
  27. 2020A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergencescited by 21
  28. 2019Separating Local & Shuffled Differential Privacy via Histogramscited by 16
  29. 2019Private Aggregation from Fewer Anonymous Messagescited by 15
  30. 2020Pure Differentially Private Summation from Anonymous Messagescited by 15
  31. 2020Shuffled Model of Federated Learning: Privacy, Communication and Accuracy Trade-offscited by 15
  32. 2020Connecting Robust Shuffle Privacy and Pan-Privacycited by 12
  33. 2017Computational Differential Privacy from Lattice-based Cryptographycited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 26; 2023: 30; 2024: 28; 2025: 16; 2026: 10.

127 papers.

Sort:

By area:

  1. 2021A Field Guide to Federated Optimizationcited by 312
  2. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  3. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  4. 2021When the Curious Abandon Honesty: Federated Learning Is Not Privatecited by 126
  5. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 81
  6. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
  7. 2021Eluding Secure Aggregation in Federated Learning via Model Inconsistencycited by 66
  8. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 64
  9. 2023Federated Continual Learning via Knowledge Fusion: A Surveycited by 53
  10. 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 52
  11. 2023Recent Advances on Federated Learning: A Systematic Surveycited by 40
  12. 2022The Fundamental Price of Secure Aggregation in Differentially Private Federated Learningcited by 39
38 more
  1. 2022Gradient Obfuscation Gives a False Sense of Security in Federated Learningcited by 37
  2. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  3. 2022Differentially Private Federated Learning with Local Regularization and Sparsificationcited by 35
  4. 2022SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compressioncited by 34
  5. 2023Make Landscape Flatter in Differentially Private Federated Learningcited by 33
  6. 2022Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacycited by 32
  7. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31
  8. 2023Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learningcited by 29
  9. 2024Teach LLMs to Phish: Stealing Private Information from Language Modelscited by 29
  10. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  11. 2022Shuffle Private Linear Contextual Banditscited by 27
  12. 2021Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errorscited by 23
  13. 2022On the (In)security of Peer-to-Peer Decentralized Machine Learningcited by 23
  14. 2023Privacy Amplification via Compression: Achieving the Optimal Privacy-Accuracy-Communication Trade-off in Distributed Mean Estimationcited by 22
  15. 2021Locally Private k-Means in One Roundcited by 19
  16. 2022How Much Privacy Does Federated Learning with Secure Aggregation Guarantee?cited by 19
  17. 2021Shuffle Private Stochastic Convex Optimizationcited by 17
  18. 2022Privacy-Aware Compression for Federated Data Analysiscited by 17
  19. 2023z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learningcited by 17
  20. 2023FedVS: Straggler-Resilient and Privacy-Preserving Vertical Federated Learning for Split Modelscited by 17
  21. 2024Efficient and Near-Optimal Noise Generation for Streaming Differential Privacycited by 17
  22. 2024The Privacy Power of Correlated Noise in Decentralized Learningcited by 17
  23. 2021On the Renyi Differential Privacy of the Shuffle Modelcited by 16
  24. 2023Reconstructing Individual Data Points in Federated Learning Hardened with Differential Privacy and Secure Aggregationcited by 16
  25. 2023Randomized Quantization is All You Need for Differential Privacy in Federated Learningcited by 16
  26. 2024Federated Learning in Practice: Reflections and Projectionscited by 15
  27. 2022Distributed Differential Privacy in Multi-Armed Banditscited by 14
  28. 2021Private Multi-Task Learning: Formulation and Applications to Federated Learningcited by 11
  29. 2022Unbounded Gradients in Federated Learning with Buffered Asynchronous Aggregationcited by 11
  30. 2024Towards Federated RLHF with Aggregated Client Preference for LLMscited by 11
  31. 2021Communication-Efficient Agnostic Federated Averagingcited by 10
  32. 2021Towards Sparse Federated Analytics: Location Heatmaps under Distributed Differential Privacy with Secure Aggregationcited by 10
  33. 2021DP-REC: Private & Communication-Efficient Federated Learningcited by 10
  34. 2024RFLPA: A Robust Federated Learning Framework against Poisoning Attacks with Secure Aggregationcited by 10
  35. 2023ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacycited by 9
  36. 2023Compression with Exact Error Distribution for Federated Learningcited by 9
  37. 2024DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptationcited by 9
  38. 2025DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Normcited by 9

50 most cited of 127 All 127 (JSON)