Position: Privacy Is a Claim, Not a Property of Synthetic Data
Not on the map. Why?
Abstract (arXiv)
Synthetic data has become a common component of machine learning research. While widely adopted, its use in privacy-sensitive contexts has quietly shifted from a claim of residual inference risk under stated assumptions to an appearance-based property inferred from data generation itself. In this position paper, we argue that this shift reflects an implicit change in community standards for what counts as sufficient privacy evidence, rather than a misunderstanding of well-established privacy principles. Drawing on an empirical analysis of recent publications across major ML venues, we show that synthetic data is frequently used in privacy-sensitive settings without explicit articulation of threat models, inference risks, or falsifiable privacy claims. As a result, privacy assurance often remains implicit, difficult to verify, and unevenly distributed, with heightened exposure for rare and minority records. We argue for treating privacy as an explicit, evidence-based scientific claim and recommend that ML venues adopt norms requiring privacy-relevant assertions to be clearly scoped, testable, and contestable.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 64 references that are in the map. First posted on arXiv 1 Sep 2026.
Builds on
25 of 64 references.
References · 25
- 2016Deep Learning with Differential Privacycited by 2,728
- 2020Extracting Training Data from Large Language Modelscited by 2,189
- 2023Detecting Pretraining Data from Large Language Modelscited by 387
- 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
- 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 175
- 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 142
- 2019Reconstruction and Membership Inference Attacks against Generative Modelscited by 122
- 2020Synthetic Data -- Anonymisation Groundhog Daycited by 108
- 2023Membership Inference Attacks against Synthetic Data through Overfitting Detectioncited by 66
- 2022TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Datacited by 57
- 2024Membership Inference Attacks against Large Vision-Language Modelscited by 33
- 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Datacited by 22
13 more
- 2024Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent Circumventioncited by 21
- 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigationcited by 19
- 2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
- 2025The DCR Delusion: Measuring the Privacy Risk of Synthetic Datacited by 11
- 2024Privacy Vulnerabilities in Marginals-based Synthetic Datacited by 10
- 2025A Consensus Privacy Metrics Framework for Synthetic Datacited by 8
- 2021Correlation inference attacks against machine learning modelscited by 7
- 2025Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Frameworkcited by 6
- 2025MIA-EPT: Membership Inference Attack via Error Prediction for Tabular Datacited by 4
- 2025Synth-MIA: A Testbed for Auditing Privacy Leakage in Tabular Data Synthesiscited by 4
- 2025Anti-Regulatory AI: How "AI Safety" is Leveraged Against Regulatory Oversightcited by 3
- 2025Why Data Anonymization Has Not Taken Offcited by 2
- 2025Membership Inference over Diffusion-models-based Synthetic Tabular Datacited by 2
Built on by
Not cited yet.