mlsanityData to 7 Oct 2026

How Private are DP-SGD Implementations?

arXiv 2403.17673 ↗First posted 26 Mar 2024cs.LG cs.CR cs.DScited by 22

Not on the map. Why?

Abstract (arXiv)

We demonstrate a substantial gap between the privacy guarantees of the Adaptive Batch Linear Queries (ABLQ) mechanism under different types of batch sampling: (i) Shuffling, and (ii) Poisson subsampling; the typical analysis of Differentially Private Stochastic Gradient Descent (DP-SGD) follows by interpreting it as a post-processing of ABLQ. While shuffling-based DP-SGD is more commonly used in practical implementations, it has not been amenable to easy privacy analysis, either analytically or even numerically. On the other hand, Poisson subsampling-based DP-SGD is challenging to scalably implement, but has a well-understood privacy analysis, with multiple open-source numerically tight privacy accountants available. This has led to a common practice of using shuffling-based DP-SGD in practice, but using the privacy analysis for the corresponding Poisson subsampling version. Our result shows that there can be a substantial gap between the privacy analysis when using the two types of batch sampling, and thus advises caution in reporting privacy parameters for DP-SGD.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

30 of 42 references.

References · 30

  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  3. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  4. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  5. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  6. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  7. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 216
  8. 2021Numerical Composition of Differential Privacycited by 174
  9. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  10. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 163
  11. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  12. 2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 121
18 more
  1. 2020DP-CGAN: Differentially Private Synthetic Data and Label Generationcited by 118
  2. 2022Differentially Private Diffusion Modelscited by 118
  3. 2020GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generatorscited by 117
  4. 2021Large-Scale Differentially Private BERTcited by 117
  5. 2018Privacy Amplification by Iterationcited by 97
  6. 2020Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shufflingcited by 96
  7. 2021Optimal Accounting of Differential Privacy via Characteristic Functioncited by 83
  8. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 71
  9. 2022Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Losscited by 60
  10. 2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
  11. 2022Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributionscited by 50
  12. 2022Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacycited by 47
  13. 2022Stronger Privacy Amplification by Shuffling for R'enyi and Approximate Differential Privacycited by 29
  14. 2022Differentially private training of residual networks with scale normalisationcited by 26
  15. 2022Faster Privacy Accounting via Evolving Discretizationcited by 14
  16. 2023A Randomized Approach for Tight Privacy Accountingcited by 11
  17. 2022Private Ad Modeling with DP-SGDcited by 6
  18. 2021A Neural Database for Differentially Private Spatial Range Queriescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 6; 2026: 7.

22 papers.

Sort:

  1. 2024Fine-Tuning Large Language Models with User-Level Differential Privacycited by 28
  2. 2024Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuningcited by 21
  3. 2024Balls-and-Bins Sampling for DP-SGDcited by 13
  4. 2025Privacy amplification by random allocationcited by 13
  5. 2024Avoiding Pitfalls for Privacy Accounting of Subsampled Mechanisms under Compositioncited by 12
  6. 2025Leveraging Randomness in Model and Data Partitioning for Privacy Amplificationcited by 7
  7. 2025Differential privacy for medical deep learning: methods, tradeoffs, and deployment implicationscited by 7
  8. 2024Convergent Privacy Loss of Noisy-SGD without Convexity and Smoothnesscited by 6
  9. 2024To Shuffle or not to Shuffle: Auditing DP-SGD with Shufflingcited by 6
  10. 2024Privacy of the last iterate in cyclically-sampled DP-SGD on nonconvex composite lossescited by 5
  11. 2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 4
  12. 2026Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGDcited by 4
10 more
  1. 2024Efficient and Scalable Implementation of Differentially Private Deep Learning without Shortcutscited by 3
  2. 2025Empirical Privacy Variancecited by 3
  3. 2026Efficient privacy loss accounting for subsampling and random allocationcited by 3
  4. 2024Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacycited by 2
  5. 2026Trade-off Functions for DP-SGD with Subsampling based on Random Allocation: Tight Upper and Lower Boundscited by 1
  6. 2025Optimizer Dynamics at the Edge of Stability with Differential Privacycited by 0
  7. 2026Rethinking Benchmarks for Differentially Private Image Classificationcited by 0
  8. 2026Private PoEtry: Private In-Context Learning via Product of Expertscited by 0
  9. 2026Refined Differentially Private Linear Regression via Extension of a Free Lunch Resultcited by 0
  10. 2026Shuffling-Aware Optimization for Private Vector Mean Estimationcited by 0