# Critical Challenges and Guidelines in Evaluating Synthetic Tabular Data: A Systematic Review

arXiv: https://arxiv.org/abs/2504.18544 · HTML: https://mlsanity.com/paper/2504.18544/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 158 references that are in the map.
- First posted on arXiv 10 Apr 2025.

## Builds on (most cited first, 26 of 26)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [DECAF: Generating Fair Synthetic Data Using Causally-Aware Generative Networks](https://mlsanity.com/paper/2110.12884/) (2021)
- [TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation](https://mlsanity.com/paper/2410.20626/) (2024)
- [Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applications](https://mlsanity.com/paper/2112.12047/) (2021)
- [Reliable Generation of Privacy-preserving Synthetic Electronic Health Record Time Series via Diffusion Models](https://mlsanity.com/paper/2310.15290/) (2023)
- [Estimating Individual Treatment Effects with Time-Varying Confounders](https://mlsanity.com/paper/2008.13620/) (2020)
- [TimeDP: Learning to Generate Multi-Domain Time Series with Domain Prompts](https://mlsanity.com/paper/2501.05403/) (2025)
- [The Health Gym: Synthetic Health-Related Datasets for the Development of Reinforcement Learning Algorithms](https://mlsanity.com/paper/2203.06369/) (2022)
- [Multi-Label Clinical Time-Series Generation via Conditional GAN](https://mlsanity.com/paper/2204.04797/) (2022)
- [Population Aware Diffusion for Time Series Generation](https://mlsanity.com/paper/2501.00910/) (2025)
- [Diffusion Models for Tabular Data Imputation and Synthetic Data Generation](https://mlsanity.com/paper/2407.02549/) (2024)
- [Conditional Generation of Medical Time Series for Extrapolation to Underrepresented Populations](https://mlsanity.com/paper/2201.08186/) (2022)
- [Measuring the quality of Synthetic data for use in competitions](https://mlsanity.com/paper/1806.11345/) (2018)
- [Synthesizing Multimodal Electronic Health Records via Predictive Diffusion Models](https://mlsanity.com/paper/2406.13942/) (2024)
- [SynEHRgy: Synthesizing Mixed-Type Structured Electronic Health Records using Decoder-Only Transformers](https://mlsanity.com/paper/2411.13428/) (2024)
- [TabRep: Training Tabular Diffusion Models with a Simple and Effective Continuous Representation](https://mlsanity.com/paper/2504.04798/) (2025)
- [Patient Flow Prediction via Discriminative Learning of Mutually-Correcting Processes](https://mlsanity.com/paper/1602.05112/) (2016)
- [MedDiffusion: Boosting Health Risk Prediction via Diffusion-based Data Augmentation](https://mlsanity.com/paper/2310.02520/) (2023)
- [Protect and Extend -- Using GANs for Synthetic Data Generation of Time-Series Medical Records](https://mlsanity.com/paper/2402.14042/) (2024)
- [Synthesizing time-series wound prognosis factors from electronic medical records using generative adversarial networks](https://mlsanity.com/paper/2105.01159/) (2021)
- [Synthetic Data Generation for Augmenting Small Samples](https://mlsanity.com/paper/2501.18741/) (2025)
- [Predicting epidemic risk from past temporal contact data](https://mlsanity.com/paper/1406.1449/) (2014)
- [Impulse data models for the inverse problem of electrocardiography](https://mlsanity.com/paper/2102.00570/) (2021)
- [Fairness-Optimized Synthetic EHR Generation for Arbitrary Downstream Predictive Tasks](https://mlsanity.com/paper/2406.02510/) (2024)

## Abstract (arXiv)

Generating synthetic tabular health data is challenging, and evaluating their quality is equally, if not more, complex. This systematic review highlights the critical importance of rigorous evaluation of synthetic health data to ensure reliability, clinical relevance, and appropriate use. From an initial identification of 2067 relevant papers published in the last ten years, 134 studies were selected for detailed analysis. Our review identifies key challenges, including lack of consensus on evaluation methods, inconsistent application of evaluation metrics, limited involvement of domain experts, inadequate reporting of dataset characteristics, and limited reproducibility of results. In response, we provide a structured consolidation of synthetic data generation and evaluation methods into taxonomies, alongside practical guidelines to support more robust and standardised evaluation practices. These findings aim to support the responsible development and use of synthetic health data, aligned with emerging expectations around transparency, reproducibility, and governance, ultimately enabling the community to fully harness its transformative potential and accelerate innovation.