# Debugging Differential Privacy: A Case Study for Privacy Auditing

arXiv: https://arxiv.org/abs/2202.12219 · HTML: https://mlsanity.com/paper/2202.12219/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 9 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 6 of 6)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [The Composition Theorem for Differential Privacy](https://mlsanity.com/paper/1311.0776/) (2013)
- [Auditing Differentially Private Machine Learning: How Private is Private SGD?](https://mlsanity.com/paper/2006.07709/) (2020)
- [Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning](https://mlsanity.com/paper/2101.04535/) (2021)
- [Variational Bayes In Private Settings (VIPS)](https://mlsanity.com/paper/1611.00340/) (2016)
- [Backpropagation Clipping for Deep Learning with Differential Privacy](https://mlsanity.com/paper/2202.05089/) (2022)

## Built on by (most cited first, 30 of 40)

- [Unlocking High-Accuracy Differentially Private Image Classification through Scale](https://mlsanity.com/paper/2204.13650/) (2022)
- [Privacy Auditing with One (1) Training Run](https://mlsanity.com/paper/2305.08846/) (2023)
- [SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning](https://mlsanity.com/paper/2212.10986/) (2022)
- [Bounding Training Data Reconstruction in DP-SGD](https://mlsanity.com/paper/2302.07225/) (2023)
- [One-shot Empirical Privacy Estimation for Federated Learning](https://mlsanity.com/paper/2302.03098/) (2023)
- [Evaluations of Machine Learning Privacy Defenses are Misleading](https://mlsanity.com/paper/2404.17399/) (2024)
- [Auditing f-Differential Privacy in One Run](https://mlsanity.com/paper/2410.22235/) (2024)
- [Selective Pre-training for Private Fine-tuning](https://mlsanity.com/paper/2305.13865/) (2023)
- [Privacy Auditing of Large Language Models](https://mlsanity.com/paper/2503.06808/) (2025)
- [Have it your way: Individualized Privacy Assignment for DP-SGD](https://mlsanity.com/paper/2303.17046/) (2023)
- [Nearly Tight Black-Box Auditing of Differentially Private Machine Learning](https://mlsanity.com/paper/2405.14106/) (2024)
- [Unleashing the Power of Randomization in Auditing Differentially Private ML](https://mlsanity.com/paper/2305.18447/) (2023)
- [Membership Inference Attacks on Diffusion Models via Quantile Regression](https://mlsanity.com/paper/2312.05140/) (2023)
- [Unlocking Accuracy and Fairness in Differentially Private Image Classification](https://mlsanity.com/paper/2308.10888/) (2023)
- [The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasets](https://mlsanity.com/paper/2312.05114/) (2023)
- [The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debugging](https://mlsanity.com/paper/2406.13985/) (2024)
- [Recent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey](https://mlsanity.com/paper/2309.16398/) (2023)
- [The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD](https://mlsanity.com/paper/2410.06186/) (2024)
- [Free Record-Level Privacy Risk Evaluation Through Artifact-Based Methods](https://mlsanity.com/paper/2411.05743/) (2024)
- [Auditing Differential Privacy Guarantees Using Density Estimation](https://mlsanity.com/paper/2406.04827/) (2024)
- [To Shuffle or not to Shuffle: Auditing DP-SGD with Shuffling](https://mlsanity.com/paper/2411.10614/) (2024)
- [Optimizing Canaries for Privacy Auditing with Metagradient Descent](https://mlsanity.com/paper/2507.15836/) (2025)
- [The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditing](https://mlsanity.com/paper/2506.16666/) (2025)
- [DeepWeightFlow: Re-Basined Flow Matching for Generating Neural Network Weights](https://mlsanity.com/paper/2601.05052/) (2026)
- [Algorithms with More Granular Differential Privacy Guarantees](https://mlsanity.com/paper/2209.04053/) (2022)
- [Enhancing One-run Privacy Auditing with Quantile Regression-Based Membership Inference](https://mlsanity.com/paper/2506.15349/) (2025)
- [Auditing Approximate Machine Unlearning for Differentially Private Models](https://mlsanity.com/paper/2508.18671/) (2025)
- [JAX-Privacy: A library for differentially private machine learning](https://mlsanity.com/paper/2602.17861/) (2026)
- [Universal Neural-Cracking-Machines: Self-Configurable Password Models from Auxiliary Data](https://mlsanity.com/paper/2301.07628/) (2023)
- [Epsilon*: Privacy Metric for Machine Learning Models](https://mlsanity.com/paper/2307.11280/) (2023)

## Abstract (arXiv)

Differential Privacy can provide provable privacy guarantees for training data in machine learning. However, the presence of proofs does not preclude the presence of errors. Inspired by recent advances in auditing which have been used for estimating lower bounds on differentially private algorithms, here we show that auditing can also be used to find flaws in (purportedly) differentially private schemes. In this case study, we audit a recent open source implementation of a differentially private deep learning algorithm and find, with 99.99999999% confidence, that the implementation does not satisfy the claimed differential privacy guarantee.