# Synthetic is all you need: removing the auxiliary data assumption for membership inference attacks against synthetic data

arXiv: https://arxiv.org/abs/2307.01701 · HTML: https://mlsanity.com/paper/2307.01701/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 13 of 13)

- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models](https://mlsanity.com/paper/1806.01246/) (2018)
- [Synthetic Data -- Anonymisation Groundhog Day](https://mlsanity.com/paper/2011.07018/) (2020)
- [Knock Knock, Who's There? Membership Inference on Aggregate Location Data](https://mlsanity.com/paper/1708.06145/) (2017)
- [TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Data](https://mlsanity.com/paper/2211.06550/) (2022)
- [A Unified Framework for Quantifying Privacy Risk in Synthetic Data](https://mlsanity.com/paper/2211.10459/) (2022)
- [Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishing](https://mlsanity.com/paper/2306.10308/) (2023)
- [A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Data](https://mlsanity.com/paper/2301.10053/) (2023)
- [Understanding Real-world Threats to Deep Learning Models in Android Apps](https://mlsanity.com/paper/2209.09577/) (2022)
- [Correlation inference attacks against machine learning models](https://mlsanity.com/paper/2112.08806/) (2021)
- [Design of a Privacy-Preserving Data Platform for Collaboration Against Human Trafficking](https://mlsanity.com/paper/2005.05688/) (2020)

## Built on by (most cited first, 18 of 18)

- [The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasets](https://mlsanity.com/paper/2312.05114/) (2023)
- [The DCR Delusion: Measuring the Privacy Risk of Synthetic Data](https://mlsanity.com/paper/2505.01524/) (2025)
- [Privacy Vulnerabilities in Marginals-based Synthetic Data](https://mlsanity.com/paper/2410.05506/) (2024)
- [Evaluating Differentially Private Generation of Domain-Specific Text](https://mlsanity.com/paper/2508.20452/) (2025)
- [A Zero Auxiliary Knowledge Membership Inference Attack on Aggregate Location Data](https://mlsanity.com/paper/2406.18671/) (2024)
- [Generating Synthetic Data with Formal Privacy Guarantees: State of the Art and the Road Ahead](https://mlsanity.com/paper/2503.20846/) (2025)
- [MIA-EPT: Membership Inference Attack via Error Prediction for Tabular Data](https://mlsanity.com/paper/2509.13046/) (2025)
- [Synthetic Data Can Mislead Evaluations: Membership Inference as Machine Text Detection](https://mlsanity.com/paper/2501.11786/) (2025)
- [VFLGAN: Vertical Federated Learning-based Generative Adversarial Network for Vertically Partitioned Data Publication](https://mlsanity.com/paper/2404.09722/) (2024)
- [Biologically-Informed Hybrid Membership Inference Attacks on Generative Genomic Models](https://mlsanity.com/paper/2511.07503/) (2025)
- [Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective](https://mlsanity.com/paper/2601.22434/) (2026)
- [A Dual Perspective on Synthetic Trajectory Generators: Utility Framework and Privacy Vulnerabilities](https://mlsanity.com/paper/2604.19653/) (2026)
- [Lost in the Averages: A New Specific Setup to Evaluate Membership Inference Attacks Against Machine Learning Models](https://mlsanity.com/paper/2405.15423/) (2024)
- [Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators](https://mlsanity.com/paper/2603.10937/) (2026)
- [Automated Membership Inference Attacks (AutoMIA): Discovering MIA Signal Computations using LLM Agents](https://mlsanity.com/paper/2603.19375/) (2026)
- [diffGHOST: Diffusion based Generative Hedged Oblivious Synthetic Trajectories](https://mlsanity.com/paper/2605.10647/) (2026)
- [ReMIA: a Powerful and Efficient Alternative to Membership Inference Attacks against Synthetic Data Generators](https://mlsanity.com/paper/2605.14686/) (2026)
- [It does what it says on the tin: safe synthetic data from coarsened margins](https://mlsanity.com/paper/2606.02101/) (2026)

## Abstract (arXiv)

Synthetic data is emerging as one of the most promising solutions to share individual-level data while safeguarding privacy. While membership inference attacks (MIAs), based on shadow modeling, have become the standard to evaluate the privacy of synthetic data, they currently assume the attacker to have access to an auxiliary dataset sampled from a similar distribution as the training dataset. This is often seen as a very strong assumption in practice, especially as the proposed main use cases for synthetic tabular data (e.g. medical data, financial transactions) are very specific and don't have any reference datasets directly available. We here show how this assumption can be removed, allowing for MIAs to be performed using only the synthetic data. Specifically, we developed three different scenarios: (S1) Black-box access to the generator, (S2) only access to the released synthetic dataset and (S3) a theoretical setup as upper bound for the attack performance using only synthetic data. Our results show that MIAs are still successful, across two real-world datasets and two synthetic data generators. These results show how the strong hypothesis made when auditing synthetic data releases - access to an auxiliary dataset - can be relaxed, making the attacks more realistic in practice.