mlsanityData to 13 Oct 2026

The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasets

arXiv 2312.05114 ↗First posted 8 Dec 2023cs.CR cs.AI cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Generative models producing synthetic data are meant to provide a privacy-friendly approach to releasing data. However, their privacy guarantees are only considered robust when models satisfy Differential Privacy (DP). Alas, this is not a ubiquitous standard, as many leading companies (and, in fact, research papers) use ad-hoc privacy metrics based on testing the statistical similarity between synthetic and real data. In this paper, we examine the privacy metrics used in real-world synthetic data deployments and demonstrate their unreliability in several ways. First, we provide counter-examples where severe privacy violations occur even if the privacy tests pass and instantiate accurate membership and attribute inference attacks with minimal cost. We then introduce ReconSyn, a reconstruction attack that generates multiple synthetic datasets that are considered private by the metrics but actually leak information unique to individual records. We show that ReconSyn recovers 78-100% of the outliers in the train data with only black-box access to a single fitted generative model and the privacy metrics. In the process, we show that applying DP only to the model does not mitigate this attack, as using privacy metrics breaks the end-to-end DP pipeline.

Key facts. Cited by 14 papers in the map (data through 13 Oct 2026). Builds on 64 of its 155 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

64 of 155 references.

References · 64

  1. 2016Deep Learning with Differential Privacycited by 2,718
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,273
  3. 2020Extracting Training Data from Large Language Modelscited by 2,181
  4. 2019Deep Leakage from Gradientscited by 1,017
  5. 2021Membership Inference Attacks From First Principlescited by 872
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  7. 2023Extracting Training Data from Diffusion Modelscited by 729
  8. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 711
  9. 2019Modeling Tabular data using Conditional GANcited by 698
  10. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 427
  11. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 376
  12. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
38 more
  1. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  2. 2022Language Models are Realistic Tabular Data Generatorscited by 292
  3. 2018Scalable Private Learning with PATEcited by 268
  4. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
  5. 2018Differentially Private Generative Adversarial Networkcited by 235
  6. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 218
  7. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 199
  8. 2018Data Synthesis based on Generative Adversarial Networkscited by 185
  9. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 184
  10. 2017Machine Learning Models that Remember Too Muchcited by 169
  11. 2021CTAB-GAN: Effective Table Data Synthesizingcited by 156
  12. 2022Reconstructing Training Data with Informed Adversariescited by 147
  13. 2022Reconstructing Training Data from Trained Neural Networkscited by 130
  14. 2019Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learningcited by 122
  15. 2019Reconstruction and Membership Inference Attacks against Generative Modelscited by 122
  16. 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 102
  17. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 94
  18. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 90
  19. 2021Benchmarking Differentially Private Synthetic Data Generation Algorithmscited by 80
  20. 2019Graphical-model based estimation and inference for differential privacycited by 73
  21. 2017Knock Knock, Who's There? Membership Inference on Aggregate Location Datacited by 69
  22. 2023Membership Inference Attacks against Synthetic Data through Overfitting Detectioncited by 66
  23. 2022TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Datacited by 57
  24. 2023Generative Table Pre-training Empowers Models for Tabular Predictioncited by 55
  25. 2021Robin Hood and Matthew Effects: Differential Privacy Has Disparate Impact on Synthetic Datacited by 48
  26. 2022A Unified Framework for Quantifying Privacy Risk in Synthetic Datacited by 46
  27. 2023Bounding Training Data Reconstruction in DP-SGDcited by 45
  28. 2019Detecting Overfitting of Deep Generative Networks via Latent Recoverycited by 44
  29. 2019Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applicationscited by 43
  30. 2022Debugging Differential Privacy: A Case Study for Privacy Auditingcited by 40
  31. 2023Privacy Side Channels in Machine Learning Systemscited by 39
  32. 2014Differential Privacy: An Economic Method for Choosing Epsiloncited by 34
  33. 2023Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishingcited by 28
  34. 2019Towards Formalizing the GDPR's Notion of Singling Outcited by 26
  35. 2023A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Datacited by 24
  36. 2023Arbitrary Decisions are a Hidden Cost of Differentially Private Trainingcited by 19
  37. 2023Synthesizing Mixed-type Electronic Health Records using Diffusion Modelscited by 18
  38. 2023Synthetic is all you need: removing the auxiliary data assumption for membership inference attacks against synthetic datacited by 18

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 7.

14 papers.

Sort:

By area:

  1. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 158
  2. 2024Systematic Assessment of Tabular Data Synthesiscited by 18
  3. 2024Data Plagiarism Index: Characterizing the Privacy Risk of Data-Copying in Tabular Generative Modelscited by 12
  4. 2025The DCR Delusion: Measuring the Privacy Risk of Synthetic Datacited by 11
  5. 2025TabRep: Training Tabular Diffusion Models with a Simple and Effective Continuous Representationcited by 5
  6. 2025Setting varepsilon is not the Issue in Differential Privacycited by 3
  7. 2025Causal-Aware Tabular GANs with Reinforcement Learningcited by 1
  8. 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspectivecited by 1
  9. 2026Where to Intervene? Benchmarking Fairness-Aware Learning on Differentially Private Synthetic Tabular Datacited by 1
  10. 2026Should I use Synthetic Data for That? An Analysis of the Suitability of Synthetic Data for Data Sharing and Augmentationcited by 0
  11. 2026Self-Improving Tabular Language Models via Iterative Reward-Guided Post-Trainingcited by 0
  12. 2026PrivFusion: A Privacy-preserving Multi-Agent Framework for Harmonizing Distributed Datasetscited by 0
2 more
  1. 2026SoK: Reconstruction Attacks on Synthetic Tabular Data (Insights from Winning the NIST CRC)cited by 0
  2. 2026When Sample Selection Bias Precipitates Model Collapsecited by 0