mlsanityData to 14 Oct 2026

A Consensus Privacy Metrics Framework for Synthetic Data

arXiv 2503.04980 ↗First posted 6 Mar 2025cs.CR cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Synthetic data generation is one approach for sharing individual-level data. However, to meet legislative requirements, it is necessary to demonstrate that the individuals' privacy is adequately protected. There is no consolidated standard for measuring privacy in synthetic data. Through an expert panel and consensus process, we developed a framework for evaluating privacy in synthetic data. Our findings indicate that current similarity metrics fail to measure identity disclosure, and their use is discouraged. For differentially private synthetic data, a privacy budget other than close to zero was not considered interpretable. There was consensus on the importance of membership and attribute disclosure, both of which involve inferring personal information about an individual without necessarily revealing their identity. The resultant framework provides precise recommendations for metrics that address these types of disclosures effectively. Our findings further present specific opportunities for future research that can help with widespread adoption of synthetic data.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 29 of its 138 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

29 of 138 references.

References · 29

  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  2. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 377
  3. 2017Generating Multi-label Discrete Patient Records using Generative Adversarial Networkscited by 197
  4. 2018Data Synthesis based on Generative Adversarial Networkscited by 185
  5. 2020Synthetic Data -- Anonymisation Groundhog Daycited by 108
  6. 2023Membership Inference Attacks against Synthetic Data through Overfitting Detectioncited by 66
  7. 2022TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Datacited by 57
  8. 2021Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applicationscited by 47
  9. 2022A Unified Framework for Quantifying Privacy Risk in Synthetic Datacited by 46
  10. 2022Geographic Spines in the 2020 Census Disclosure Avoidance Systemcited by 44
  11. 2022A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Modelscited by 43
  12. 2020CorGAN: Correlation-Capturing Convolutional Generative Adversarial Networks for Generating Synthetic Healthcare Recordscited by 37
17 more
  1. 2020Differentially Private Synthetic Data: Applied Evaluations and Enhancementscited by 36
  2. 2023Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishingcited by 28
  3. 2024Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcarecited by 17
  4. 2024Differentially Private Release of Israel's National Registry of Live Birthscited by 17
  5. 2021Fidelity and Privacy of Synthetic Medical Datacited by 16
  6. 2022A review of Generative Adversarial Networks for Electronic Health Records: applications, evaluation measures and data sourcescited by 13
  7. 2024Performance evaluation of predictive AI models to support medical decisions: Overview and guidancecited by 9
  8. 2021On Utility and Privacy in Synthetic Genomic Datacited by 8
  9. 202330 Years of Synthetic Datacited by 7
  10. 2022An Empirical Study on the Membership Inference Attack against Tabular Data Synthesis Modelscited by 6
  11. 2020A Members First Approach to Enabling LinkedIn's Labor Market Insights at Scalecited by 5
  12. 2023Database Reconstruction Is Not So Easy and Is Different from Reidentificationcited by 4
  13. 2023Privacy Measurement in Tabular Synthetic Data: State of the Art and Future Research Directionscited by 4
  14. 2022A Note on the Misinterpretation of the US Census Re-identification Attackcited by 2
  15. 2024Selecting a classification performance measure: matching the measure to the problemcited by 2
  16. 2024Towards more accurate and useful data anonymity vulnerability measurescited by 1
  17. 2025Synthetic Data and Health Privacycited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025Privacy-Preserving Tabular Synthetic Data Generation Using TabularARGNcited by 0
  2. 2025Synthetic data for ratemaking: imputation-based methods vs adversarial networks and autoencoderscited by 0
  3. 2026SynQP: A Framework and Metrics for Evaluating the Quality and Privacy Risk of Synthetic Datacited by 0
  4. 2026Evaluation metrics for temporal preservation in synthetic longitudinal patient datacited by 0
  5. 2026A Critical Review on the Effectiveness and Privacy Threats of Membership Inference Attackscited by 0
  6. 2026Not all Jensen-Shannon Divergence Estimators are Equalcited by 0
  7. 2026Generating Benchmark Health Data Using a Tabular Diffusion Transformercited by 0
  8. 2026Position: Privacy Is a Claim, Not a Property of Synthetic Datacited by 0