# "What do you want from theory alone?" Experimenting with Tight Auditing of Differentially Private Synthetic Data Generation

arXiv: https://arxiv.org/abs/2405.10994 · HTML: https://mlsanity.com/paper/2405.10994/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2024.

## Built on by (most cited first, 21 of 21)

- [The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debugging](https://mlsanity.com/paper/2406.13985/) (2024)
- [DPImageBench: A Unified Benchmark for Differentially Private Image Synthesis](https://mlsanity.com/paper/2503.14681/) (2025)
- [The DCR Delusion: Measuring the Privacy Risk of Synthetic Data](https://mlsanity.com/paper/2505.01524/) (2025)
- [Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective](https://mlsanity.com/paper/2506.13009/) (2025)
- [To Shuffle or not to Shuffle: Auditing DP-SGD with Shuffling](https://mlsanity.com/paper/2411.10614/) (2024)
- [The Importance of Being Discrete: Measuring the Impact of Discretization in End-to-End Differentially Private Synthetic Data](https://mlsanity.com/paper/2504.06923/) (2025)
- [The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditing](https://mlsanity.com/paper/2506.16666/) (2025)
- [dpmm: Differentially Private Marginal Models, a Library for Synthetic Tabular Data Generation](https://mlsanity.com/paper/2506.00322/) (2025)
- [Privacy Amplification Through Synthetic Data: Insights from Linear Regression](https://mlsanity.com/paper/2506.05101/) (2025)
- [Finding Connections: Membership Inference Attacks for the Multi-Table Synthetic Data Setting](https://mlsanity.com/paper/2602.07126/) (2026)
- [Synthetic Data, Similarity-based Privacy Metrics, and Regulatory (Non-)Compliance](https://mlsanity.com/paper/2407.16929/) (2024)
- [Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective](https://mlsanity.com/paper/2601.22434/) (2026)
- [Tight Auditing of Differential Privacy in MST and AIM](https://mlsanity.com/paper/2604.18352/) (2026)
- [Lost in the Averages: A New Specific Setup to Evaluate Membership Inference Attacks Against Machine Learning Models](https://mlsanity.com/paper/2405.15423/) (2024)
- [Understanding the Impact of Data Domain Extraction on Synthetic Data Privacy](https://mlsanity.com/paper/2504.08254/) (2025)
- [DeSIA: Attribute Inference Attacks Against Limited Fixed Aggregate Statistics](https://mlsanity.com/paper/2504.18497/) (2025)
- [Privacy Amplification Persists under Unlimited Synthetic Data Release](https://mlsanity.com/paper/2602.04895/) (2026)
- [On Privacy in Data-Space Tabular Diffusion Models: Influential Factors, Attacker Knowledge, and Metrics](https://mlsanity.com/paper/2605.06835/) (2026)
- [PrivFusion: A Privacy-preserving Multi-Agent Framework for Harmonizing Distributed Datasets](https://mlsanity.com/paper/2605.24249/) (2026)
- [Canaries in the Bank: Auditing User-Level Privacy in Private Evolution](https://mlsanity.com/paper/2609.13499/) (2026)
- [Tight End-to-End Accounting for Differentially Private Synthetic Data](https://mlsanity.com/paper/2610.17249/) (2026)

## Abstract (arXiv)

Differentially private synthetic data generation (DP-SDG) algorithms are used to release datasets that are structurally and statistically similar to sensitive data while providing formal bounds on the information they leak. However, bugs in algorithms and implementations may cause the actual information leakage to be higher. This prompts the need to verify whether the theoretical guarantees of state-of-the-art DP-SDG implementations also hold in practice. We do so via a rigorous auditing process: we compute the information leakage via an adversary playing a distinguishing game and running membership inference attacks (MIAs). If the leakage observed empirically is higher than the theoretical bounds, we identify a DP violation; if it is non-negligibly lower, the audit is loose. We audit six DP-SDG implementations using different datasets and threat models and find that black-box MIAs commonly used against DP-SDGs are severely limited in power, yielding remarkably loose empirical privacy estimates. We then consider MIAs in stronger threat models, i.e., passive and active white-box, using both existing and newly proposed attacks. Overall, we find that, currently, we do not only need white-box MIAs but also worst-case datasets to tightly estimate the privacy leakage from DP-SDGs. Finally, we show that our automated auditing procedure finds both known DP violations (in 4 out of the 6 implementations) as well as a new one in the DPWGAN implementation that was successfully submitted to the NIST DP Synthetic Data Challenge. The source code needed to reproduce our experiments is available from https://github.com/spalabucr/synth-audit.