mlsanityData to 7 Oct 2026

Synthetic Data Privacy Metrics

arXiv 2501.03941 ↗First posted 7 Jan 2025cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent advancements in generative AI have made it possible to create synthetic datasets that can be as accurate as real-world data for training AI models, powering statistical insights, and fostering collaboration with sensitive datasets while offering strong privacy guarantees. Effectively measuring the empirical privacy of synthetic data is an important step in the process. However, while there is a multitude of new privacy metrics being published every day, there currently is no standardization. In this paper, we review the pros and cons of popular metrics that include simulations of adversarial attacks. We also review current best practices for amending generative models to enhance the privacy of the data they create (e.g. differential privacy).

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 50 of its 95 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

50 of 95 references.

References · 50

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  2. 2020Extracting Training Data from Large Language Modelscited by 2,144
  3. 2021Membership Inference Attacks From First Principlescited by 839
  4. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  5. 2023Extracting Training Data from Diffusion Modelscited by 723
  6. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  7. 2018ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Modelscited by 475
  8. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  9. 2020Label-Only Membership Inference Attackscited by 371
  10. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  11. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  12. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
38 more
  1. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  2. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  3. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  4. 2017Generating Multi-label Discrete Patient Records using Generative Adversarial Networkscited by 197
  5. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  6. 2024Do Membership Inference Attacks Work on Large Language Models?cited by 188
  7. 2018Data Synthesis based on Generative Adversarial Networkscited by 184
  8. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 183
  9. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  10. 2020Membership Leakage in Label-Only Exposurescited by 156
  11. 2021CTAB-GAN: Effective Table Data Synthesizingcited by 156
  12. 2021On the Importance of Difficulty Calibration in Membership Inference Attackscited by 137
  13. 2023Are Diffusion Models Vulnerable to Membership Inference Attacks?cited by 135
  14. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  15. 2019Reconstruction and Membership Inference Attacks against Generative Modelscited by 122
  16. 2023Privacy Auditing with One (1) Training Runcited by 119
  17. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 108
  18. 2020Synthetic Data -- Anonymisation Groundhog Daycited by 105
  19. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  20. 2022TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Datacited by 56
  21. 2021Membership Inference Attack Susceptibility of Clinical Language Modelscited by 50
  22. 2021Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applicationscited by 47
  23. 2022A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Modelscited by 43
  24. 2023Privacy Side Channels in Machine Learning Systemscited by 38
  25. 2023User Inference Attacks on Large Language Modelscited by 36
  26. 2024On the Privacy Risk of In-context Learningcited by 31
  27. 2019Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentationcited by 29
  28. 2020Investigating Membership Inference Attacks under Data Dependenciescited by 27
  29. 2018Performing Co-Membership Attacks Against Deep Generative Modelscited by 24
  30. 2023A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Datacited by 23
  31. 2020Generating Electronic Health Records with Multiple Data Types and Constraintscited by 21
  32. 2024Scaling While Privacy Preserving: A Comprehensive Synthetic Tabular Data Generation and Evaluation in Learning Analyticscited by 18
  33. 2024Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcarecited by 17
  34. 2021Fidelity and Privacy of Synthetic Medical Datacited by 16
  35. 2022A review of Generative Adversarial Networks for Electronic Health Records: applications, evaluation measures and data sourcescited by 13
  36. 2023Assessing Privacy Risks in Language Models: A Case Study on Summarization Taskscited by 8
  37. 2024DP-TLDM: Differentially Private Tabular Latent Diffusion Modelcited by 5
  38. 2024Closed-Form Bounds for DP-SGD against Record-level Inferencecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025Graph-Conditional Flow Matching for Relational Data Generationcited by 3
  2. 2025ICAFS: Inter-Client-Aware Feature Selection for Vertical Federated Learningcited by 0
  3. 2025Membership Inference Attack with Partial Featurescited by 0
  4. 2025Limited Reference, Reliable Generation: A Two-Component Framework for Tabular Data Generation in Low-Data Regimescited by 0
  5. 2025Toward Valid Generative Clinical Trial Data with Survival Endpointscited by 0
  6. 2026Controllable Generative Sandbox for Causal Inferencecited by 0
  7. 2026Evaluating LLM Simulators as Differentially Private Data Generatorscited by 0