mlsanityData to 7 Oct 2026

Avoiding Pitfalls for Privacy Accounting of Subsampled Mechanisms under Composition

arXiv 2405.20769 ↗First posted 27 May 2024cs.CR cs.DS cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

We consider the problem of computing tight privacy guarantees for the composition of subsampled differentially private mechanisms. Recent algorithms can numerically compute the privacy parameters to arbitrary precision but must be carefully applied. Our main contribution is to address two common points of confusion. First, some privacy accountants assume that the privacy guarantees for the composition of a subsampled mechanism are determined by self-composing the worst-case datasets for the uncomposed mechanism. We show that this is not true in general. Second, Poisson subsampling is sometimes assumed to have similar privacy guarantees compared to sampling without replacement. We show that the privacy guarantees may in fact differ significantly between the two sampling schemes. In particular, we give an example of hyperparameters that result in varepsilon ≈ 1 for Poisson subsampling and varepsilon > 10 for sampling without replacement. This occurs for some parameters that could realistically be chosen for DP-SGD.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

18 of 27 references.

References · 18

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  3. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  4. 2013The Composition Theorem for Differential Privacycited by 290
  5. 2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 237
  6. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 234
  7. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  8. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 216
  9. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  10. 2021Numerical Composition of Differential Privacycited by 174
  11. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  12. 2016Concentrated Differential Privacycited by 139
6 more
  1. 2021Optimal Accounting of Differential Privacy via Characteristic Functioncited by 83
  2. 2022Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributionscited by 50
  3. 2020Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFTcited by 43
  4. 2022Composition of Differential Privacy & Privacy Amplification by Subsamplingcited by 42
  5. 2024How Private are DP-SGD Implementations?cited by 22
  6. 2023A Randomized Approach for Tight Privacy Accountingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 4.

12 papers.

  1. 2024Scalable DP-SGD: Shuffling vs. Poisson Subsamplingcited by 16
  2. 2024Balls-and-Bins Sampling for DP-SGDcited by 13
  3. 2025Privacy amplification by random allocationcited by 13
  4. 2024Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacementcited by 7
  5. 2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 4
  6. 2024Efficient and Scalable Implementation of Differentially Private Deep Learning without Shortcutscited by 3
  7. 2026Sampling-Free Privacy Accounting for Matrix Mechanisms under Random Allocationcited by 3
  8. 2024Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacycited by 2
  9. 2026DP-λCGD: Efficient Noise Correlation for Differentially Private Model Trainingcited by 1
  10. 2025Private Rate-Constrained Optimization with Applications to Fair Learningcited by 0
  11. 2026Rethinking Benchmarks for Differentially Private Image Classificationcited by 0
  12. 2026Dithered Gaussian Mechanism for Randomness-Efficient Differential Privacycited by 0