mlsanityData to 14 Oct 2026

Can I trust my fake data -- A comprehensive quality assessment framework for synthetic tabular data in healthcare

arXiv 2401.13716 ↗First posted 24 Jan 2024cs.LG cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

Ensuring safe adoption of AI tools in healthcare hinges on access to sufficient data for training, testing and validation. In response to privacy concerns and regulatory requirements, using synthetic data has been suggested. Synthetic data is created by training a generator on real data to produce a dataset with similar statistical properties. Competing metrics with differing taxonomies for quality evaluation have been suggested, resulting in a complex landscape. Optimising quality entails balancing considerations that make the data fit for use, yet relevant dimensions are left out of existing frameworks. We performed a comprehensive literature review on the use of quality evaluation metrics on SD within the scope of tabular healthcare data and SD made using deep generative methods. Based on this and the collective team experiences, we developed a conceptual framework for quality assurance. The applicability was benchmarked against a practical case from the Dutch National Cancer Registry. We present a conceptual framework for quality assurance of SD for AI applications in healthcare that aligns diverging taxonomies, expands on common quality dimensions to include the dimensions of Fairness and Carbon footprint, and proposes stages necessary to support real-life applications. Building trust in synthetic data by increasing transparency and reducing the safety risk will accelerate the development and uptake of trustworthy AI tools for the benefit of patients. Despite the growing emphasis on algorithmic fairness and carbon footprint, these metrics were scarce in the literature review. The overwhelming focus was on statistical similarity using distance metrics while sequential logic detection was scarce. A consensus-backed framework that includes all relevant quality dimensions can provide assurance for safe and responsible real-life applications of SD.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 24 of its 138 references that are in the map. First posted on arXiv 24 Jan 2024.

Builds on

24 of 138 references.

References · 24

  1. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  2. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  3. 2021How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Modelscited by 196
  4. 2020Differentially Private Synthetic Medical Data Generation using Convolutional GANscited by 52
  5. 2021Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applicationscited by 47
  6. 2022A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Modelscited by 43
  7. 2023Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learningcited by 42
  8. 2020CorGAN: Correlation-Capturing Convolutional Generative Adversarial Networks for Generating Synthetic Healthcare Recordscited by 37
  9. 2023Beyond Privacy: Navigating the Opportunities and Challenges of Synthetic Datacited by 29
  10. 2023Synthetic data, real errors: how (not) to publish and use synthetic datacited by 29
  11. 2022TabSynDex: A Universal Metric for Robust Evaluation of Synthetic Tabular Datacited by 27
  12. 2020Longitudinal Variational Autoencodercited by 25
12 more
  1. 2020Generating Electronic Health Records with Multiple Data Types and Constraintscited by 21
  2. 2022The Health Gym: Synthetic Health-Related Datasets for the Development of Reinforcement Learning Algorithmscited by 19
  3. 2023SurvivalGAN: Generating Time-to-Event Data for Survival Analysiscited by 17
  4. 2022Multi-Label Clinical Time-Series Generation via Conditional GANcited by 16
  5. 2021Transitioning from Real to Synthetic data: Quantifying the bias in modelcited by 12
  6. 2022Generating Synthetic Clinical Data that Capture Class Imbalanced Distributions with Generative Adversarial Networks: Example using Antiretroviral Therapy for HIVcited by 12
  7. 2020Really Useful Synthetic Data -- A Framework to Evaluate the Quality of Differentially Private Synthetic Datacited by 11
  8. 2021On Utility and Privacy in Synthetic Genomic Datacited by 8
  9. 2020Deep generative models in DataSHIELDcited by 3
  10. 2022Synthetic data enable experiments in atomistic machine learningcited by 3
  11. 2019Precision-Recall Curves Using Information Divergence Frontierscited by 1
  12. 2023Predicting Visit Cost of Obstructive Sleep Apnea using Electronic Healthcare Records with Transformercited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 10; 2026: 6.

17 papers.

Sort:

By area:

  1. 2025Synthetic Data Privacy Metricscited by 8
  2. 2025A Consensus Privacy Metrics Framework for Synthetic Datacited by 8
  3. 2025Ensembling Membership Inference Attacks Against Tabular Generative Modelscited by 4
  4. 2025A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoptioncited by 3
  5. 2025When Tables Leak: Attacking String Memorization in LLM-Based Tabular Data Generationcited by 3
  6. 2025SHAP Distance: An Explainability-Aware Metric for Evaluating the Semantic Fidelity of Synthetic Tabular Datacited by 1
  7. 2026SAGE: Sparse Adaptive Guidance for Dependency-Aware Tabular Data Generationcited by 1
  8. 2026Rethinking Data Augmentation under Covariate Shift: Invariant-Guided Diffusion and Prototype Reweightingcited by 1
  9. 2024Permissioned Blockchain-based Framework for Ranking Synthetic Data Generatorscited by 0
  10. 2025Opinion: Revisiting synthetic data classifications from a privacy perspectivecited by 0
  11. 2025A Statistical Approach for Synthetic EEG Data Generationcited by 0
  12. 2025Generating Reliable Synthetic Clinical Trial Data: The Role of Hyperparameter Optimization and Domain Constraintscited by 0
5 more
  1. 2025Synthetic medical data generation: state of the art and application to trauma mechanism classificationcited by 0
  2. 2026Evaluation metrics for temporal preservation in synthetic longitudinal patient datacited by 0
  3. 2026Evaluating quality in synthetic data generation for large tabular health datasetscited by 0
  4. 2026Synthetic but Not Realistic: The Evaluation Challenge in Generative Modelling for Structured Electronic Medical Recordscited by 0
  5. 2026FairDiffuseVQVAE: Sampling-Time Fairness in Tabular Diffusion via Conditional Refinement of Vector-Quantized Latentscited by 0