# A Unified Framework for Quantifying Privacy Risk in Synthetic Data

arXiv: https://arxiv.org/abs/2211.10459 · HTML: https://mlsanity.com/paper/2211.10459/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2022.

## Built on by (most cited first, 30 of 46)

- [Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishing](https://mlsanity.com/paper/2306.10308/) (2023)
- [A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Data](https://mlsanity.com/paper/2301.10053/) (2023)
- [Synthetic is all you need: removing the auxiliary data assumption for membership inference attacks against synthetic data](https://mlsanity.com/paper/2307.01701/) (2023)
- [Systematic Assessment of Tabular Data Synthesis](https://mlsanity.com/paper/2402.06806/) (2024)
- [The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasets](https://mlsanity.com/paper/2312.05114/) (2023)
- [SiloFuse: Cross-silo Synthetic Data Generation with Latent Tabular Diffusion Models](https://mlsanity.com/paper/2404.03299/) (2024)
- [A Consensus Privacy Metrics Framework for Synthetic Data](https://mlsanity.com/paper/2503.04980/) (2025)
- [Synthetic Data: Revisiting the Privacy-Utility Trade-off](https://mlsanity.com/paper/2407.07926/) (2024)
- [Struct-Bench: A Benchmark for Differentially Private Structured Text Generation](https://mlsanity.com/paper/2509.10696/) (2025)
- [Diverse Community Data for Benchmarking Data Privacy Algorithms](https://mlsanity.com/paper/2306.13216/) (2023)
- [Tabular Data Synthesis with Differential Privacy: A Survey](https://mlsanity.com/paper/2411.03351/) (2024)
- [DP-TLDM: Differentially Private Tabular Latent Diffusion Model](https://mlsanity.com/paper/2403.07842/) (2024)
- [BadGD: A unified data-centric framework to identify gradient descent vulnerabilities](https://mlsanity.com/paper/2405.15979/) (2024)
- [Differentially-Private Data Synthetisation for Efficient Re-Identification Risk Control](https://mlsanity.com/paper/2212.00484/) (2022)
- [Privacy Measurement in Tabular Synthetic Data: State of the Art and Future Research Directions](https://mlsanity.com/paper/2311.17453/) (2023)
- [Synth-MIA: A Testbed for Auditing Privacy Leakage in Tabular Data Synthesis](https://mlsanity.com/paper/2509.18014/) (2025)
- [When Synthetic Data Met Regulation](https://mlsanity.com/paper/2307.00359/) (2023)
- [From Principle to Practice: Vertical Data Minimization for Machine Learning](https://mlsanity.com/paper/2311.10500/) (2023)
- [Synthetic Data Outliers: Navigating Identity Disclosure](https://mlsanity.com/paper/2406.02736/) (2024)
- [Synthetic Data Aided Federated Learning Using Foundation Models](https://mlsanity.com/paper/2407.05174/) (2024)
- [Quantifying the Privacy Implications of High-Fidelity Synthetic Network Traffic](https://mlsanity.com/paper/2511.20497/) (2025)
- [Federated Diffusion Modeling with Differential Privacy for Tabular Data Synthesis](https://mlsanity.com/paper/2412.16083/) (2024)
- [Contrastive Learning-Based privacy metrics in Tabular Synthetic Datasets](https://mlsanity.com/paper/2502.13833/) (2025)
- [MIDST Challenge at SaTML 2025: Membership Inference over Diffusion-models-based Synthetic Tabular data](https://mlsanity.com/paper/2603.19185/) (2026)
- [On the Challenges of Deploying Privacy-Preserving Synthetic Data in the Enterprise](https://mlsanity.com/paper/2307.04208/) (2023)
- [Towards more accurate and useful data anonymity vulnerability measures](https://mlsanity.com/paper/2403.06595/) (2024)
- [Six Levels of Privacy: A Framework for Financial Synthetic Data](https://mlsanity.com/paper/2403.14724/) (2024)
- [VFLGAN: Vertical Federated Learning-based Generative Adversarial Network for Vertically Partitioned Data Publication](https://mlsanity.com/paper/2404.09722/) (2024)
- [Privacy-hardened and hallucination-resistant synthetic data generation with logic-solvers](https://mlsanity.com/paper/2410.16705/) (2024)
- [TAMIS: Tailored Membership Inference Attacks on Synthetic Data](https://mlsanity.com/paper/2504.00758/) (2025)

## Abstract (arXiv)

Synthetic data is often presented as a method for sharing sensitive information in a privacy-preserving manner by reproducing the global statistical properties of the original data without disclosing sensitive information about any individual. In practice, as with other anonymization methods, privacy risks cannot be entirely eliminated. The residual privacy risks need instead to be ex-post assessed. We present Anonymeter, a statistical framework to jointly quantify different types of privacy risks in synthetic tabular datasets. We equip this framework with attack-based evaluations for the singling out, linkability, and inference risks, the three key indicators of factual anonymization according to the European General Data Protection Regulation (GDPR). To the best of our knowledge, we are the first to introduce a coherent and legally aligned evaluation of these three privacy risks for synthetic data, and to design privacy attacks which model directly the singling out and linkability risks. We demonstrate the effectiveness of our methods by conducting an extensive set of experiments that measure the privacy risks of data with deliberately inserted privacy leakages, and of synthetic data generated with and without differential privacy. Our results highlight that the three privacy risks reported by our framework scale linearly with the amount of privacy leakage in the data. Furthermore, we observe that synthetic data exhibits the lowest vulnerability against linkability, indicating one-to-one relationships between real and synthetic data records are not preserved. Finally, we demonstrate quantitatively that Anonymeter outperforms existing synthetic data privacy evaluation frameworks both in terms of detecting privacy leaks, as well as computation speed. To contribute to a privacy-conscious usage of synthetic data, we open source Anonymeter at https://github.com/statice/anonymeter.