mlsanityData to 7 Oct 2026

R'enyi Differential Privacy of the Sampled Gaussian Mechanism

arXiv 1908.10530 ↗First posted 28 Aug 2019cs.LG cs.CR stat.MLcited by 199

Not on the map. Why?

Abstract (arXiv)

The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadratically, rather than linearly, with the sampling rate. Characterizing the precise privacy properties of SGM motivated development of several relaxations of the notion of differential privacy. This work unifies and fills in gaps in published results on SGM. We describe a numerically stable procedure for precise computation of SGM's R'enyi Differential Privacy and prove a nearly tight (within a small constant factor) closed-form bound.

Key facts. Cited by 199 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 37; 2023: 43; 2024: 33; 2025: 40; 2026: 15.

199 papers.

Sort:

By area:

  1. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  2. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  3. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  4. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  5. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  6. 2022Reconstructing Training Data with Informed Adversariescited by 143
  7. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  8. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  9. 2023Privacy Auditing with One (1) Training Runcited by 119
  10. 2022Differentially Private Diffusion Modelscited by 118
  11. 2021Hyperparameter Tuning with Renyi Differential Privacycited by 104
  12. 2021Large Scale Private Learning via Low-rank Reparametrizationcited by 87
38 more
  1. 2021Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learningcited by 86
  2. 2021The Skellam Mechanism for Differentially Private Federated Learningcited by 66
  3. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
  4. 2021Decentralized Federated Learning through Proxy Model Sharingcited by 56
  5. 2022TAN Without a Burn: Scaling Laws of DP-SGDcited by 51
  6. 2023(Amplified) Banded Matrix Factorization: A unified approach to private trainingcited by 49
  7. 2022Differentially Private Bias-Term Fine-tuning of Foundation Modelscited by 47
  8. 2022Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)cited by 45
  9. 2022Bounding Training Data Reconstruction in Private (Deep) Learningcited by 43
  10. 2022Composition of Differential Privacy & Privacy Amplification by Subsamplingcited by 42
  11. 2022Large Scale Transfer Learning for Differentially Private Image Classificationcited by 40
  12. 2022Private Adaptive Optimization with Side Informationcited by 38
  13. 2022On Privacy and Personalization in Cross-Silo Federated Learningcited by 36
  14. 2022Normalized/Clipped SGD with Perturbation for Differentially Private Non-Convex Optimizationcited by 35
  15. 2022Private Set Generation with Discriminative Informationcited by 34
  16. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31
  17. 2022Disparate Impact in Differential Privacy from Gradient Misalignmentcited by 31
  18. 2022Defending against Reconstruction Attacks with R'enyi Differential Privacycited by 30
  19. 2023DP-AdamBC: Your DP-Adam Is Actually DP-SGD (Unless You Apply Bias Correction)cited by 28
  20. 2024Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domainscited by 27
  21. 2022CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learningcited by 24
  22. 2021Not all noise is accounted equally: How differentially private learning benefits from large sampling ratescited by 21
  23. 2023Have it your way: Individualized Privacy Assignment for DP-SGDcited by 21
  24. 2021Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFTcited by 20
  25. 2021DPNAS: Neural Architecture Search for Deep Learning with Differential Privacycited by 20
  26. 2022Individual Privacy Accounting for Differentially Private Stochastic Gradient Descentcited by 20
  27. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  28. 2024Privacy Backdoors: Stealing Data with Corrupted Pretrained Modelscited by 20
  29. 2022Individual Privacy Accounting with Gaussian Differential Privacycited by 19
  30. 2023Practical Differentially Private Hyperparameter Tuning with Subsamplingcited by 19
  31. 2022Differentially Private Adaptive Optimization with Delayed Preconditionerscited by 18
  32. 2023Practical Differentially Private and Byzantine-resilient Federated Learningcited by 18
  33. 2021Wide Network Learning with Differential Privacycited by 17
  34. 2022Distribution inference risks: Identifying and mitigating sources of leakagecited by 17
  35. 2023z-SignFedAvg: A Unified Stochastic Sign-based Compression for Federated Learningcited by 17
  36. 2023Private GANs, Revisitedcited by 17
  37. 2023ViP: A Differentially Private Foundation Model for Computer Visioncited by 17
  38. 2023Privacy Amplification for Matrix Mechanismscited by 17

50 most cited of 199 All 199 (JSON)