mlsanityData to 14 Oct 2026

Nearly Tight Black-Box Auditing of Differentially Private Machine Learning

arXiv 2405.14106 ↗First posted 23 May 2024cs.CR cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

This paper presents an auditing procedure for the Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm in the black-box threat model that is substantially tighter than prior work. The main intuition is to craft worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters. For models trained on MNIST and CIFAR-10 at theoretical varepsilon=10.0, our auditing procedure yields empirical estimates of varepsilonemp = 7.21 and 6.95, respectively, on a 1,000-record sample and varepsilonemp= 6.48 and 4.96 on the full datasets. By contrast, previous audits were only (relatively) tight in stronger white-box models, where the adversary can access the model's inner parameters and insert arbitrary gradients. Overall, our auditing procedure can offer valuable insight into how the privacy analysis of DP-SGD could be improved and detect bugs and DP violations in real-world implementations. The source code needed to reproduce our experiments is available at https://github.com/spalabucr/bb-audit-dpsgd.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

26 of 45 references.

References · 26

  1. 2016Wide Residual Networkscited by 3,070
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  4. 2021Membership Inference Attacks From First Principlescited by 878
  5. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
  6. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 271
  7. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 248
  8. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 235
  9. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 224
  10. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 218
  11. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 199
  12. 2023Privacy Auditing with One (1) Training Runcited by 123
14 more
  1. 2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 122
  2. 2020Synthetic Data -- Anonymisation Groundhog Daycited by 108
  3. 2021Differential Privacy Dynamics of Langevin Diffusion and Noisy Gradient Descentcited by 73
  4. 2022Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)cited by 48
  5. 2022Geographic Spines in the 2020 Census Disclosure Avoidance Systemcited by 44
  6. 2022Debugging Differential Privacy: A Case Study for Privacy Auditingcited by 40
  7. 2023One-shot Empirical Privacy Estimation for Federated Learningcited by 36
  8. 2022Bayesian Estimation of Differential Privacycited by 33
  9. 2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 30
  10. 2022CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learningcited by 24
  11. 2024PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Trapscited by 23
  12. 2024Privacy Backdoors: Stealing Data with Corrupted Pretrained Modelscited by 22
  13. 2021Not all noise is accounted equally: How differentially private learning benefits from large sampling ratescited by 21
  14. 2023Pool Inference Attacks on Local Differential Privacy: Quantifying the Privacy Guarantees of Apple's Count Mean Sketch in Practicecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 10; 2026: 5.

21 papers.

Sort:

By area:

  1. 2025Privacy Auditing of Large Language Modelscited by 23
  2. 2024The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debuggingcited by 13
  3. 2024The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGDcited by 11
  4. 2025DPImageBench: A Unified Benchmark for Differentially Private Image Synthesiscited by 11
  5. 2024It's Our Loss: No Privacy Amplification for Hidden State DP-SGD With Non-Convex Losscited by 10
  6. 2024Auditing Differential Privacy Guarantees Using Density Estimationcited by 6
  7. 2024To Shuffle or not to Shuffle: Auditing DP-SGD with Shufflingcited by 6
  8. 2025Optimizing Canaries for Privacy Auditing with Metagradient Descentcited by 6
  9. 2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 5
  10. 2025Enhancing One-run Privacy Auditing with Quantile Regression-Based Membership Inferencecited by 4
  11. 2025Empirical Privacy Variancecited by 3
  12. 2025A Unified Framework for Adversary-Aware Differential Privacy Boundscited by 2
9 more
  1. 2026Understanding Disclosure Risk in Differential Privacy with Applications to Noise Calibration and Auditing (Extended Version)cited by 2
  2. 2026Auditing of Unlearning Algorithmscited by 1
  3. 2024Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarioscited by 0
  4. 2025Sublinear Algorithms for Wasserstein and Total Variation Distances: Applications to Fairness and Privacy Auditingcited by 0
  5. 2025On the Fairness of Privacy Protection: Measuring and Mitigating the Disparity of Group Privacy Risks for Differentially Private Machine Learningcited by 0
  6. 2025Beyond Membership: Limitations of Add/Remove Adjacency in Differential Privacycited by 0
  7. 2026One RNG to Rule Them All: How Randomness Becomes an Attack Vector in Machine Learningcited by 0
  8. 2026Optimal Guarantees for Auditing R'enyi Differentially Private Machine Learningcited by 0
  9. 2026Let's Ask Gauss: Improved One-Run Privacy Auditingcited by 0