# Generative AI for Synthetic Data Across Multiple Medical Modalities: A Systematic Review of Recent Developments and Challenges

arXiv: https://arxiv.org/abs/2407.00116 · HTML: https://mlsanity.com/paper/2407.00116/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Medical synthetic data and agents, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 62 of its 261 references that are in the map.
- First posted on arXiv 27 Jun 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Medical synthetic data and agents](https://mlsanity.com/synth/medical-synthetic-data-and-agents/)

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [SRFlow: Learning the Super-Resolution Space with Normalizing Flow](https://mlsanity.com/paper/2006.14200/) (2020)
- [Generating Multi-label Discrete Patient Records using Generative Adversarial Networks](https://mlsanity.com/paper/1703.06490/) (2017)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [Diffusion Probabilistic Models beat GANs on Medical Images](https://mlsanity.com/paper/2212.07501/) (2022)
- [GANs for Medical Image Synthesis: An Empirical Study](https://mlsanity.com/paper/2105.05318/) (2021)
- [Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN](https://mlsanity.com/paper/2008.01910/) (2020)
- [Differentially Private Synthetic Medical Data Generation using Convolutional GANs](https://mlsanity.com/paper/2012.11774/) (2020)
- [Deep Learning Approaches for Data Augmentation in Medical Imaging: A Review](https://mlsanity.com/paper/2307.13125/) (2023)
- [Generating Synthetic Mixed-type Longitudinal Electronic Health Records for Artificial Intelligent Applications](https://mlsanity.com/paper/2112.12047/) (2021)
- [Causal Recurrent Variational Autoencoder for Medical Time Series Generation](https://mlsanity.com/paper/2301.06574/) (2023)
- [Sketch guided and progressive growing GAN for realistic and editable ultrasound image synthesis](https://mlsanity.com/paper/2204.06929/) (2022)
- [TTS-GAN: A Transformer-based Time-Series Generative Adversarial Network](https://mlsanity.com/paper/2202.02691/) (2022)
- [Overcoming Barriers to Data Sharing with Medical Image Generation: A Comprehensive Evaluation](https://mlsanity.com/paper/2012.03769/) (2020)
- [Diffusion-based Conditional ECG Generation with Structured State Space Models](https://mlsanity.com/paper/2301.08227/) (2023)
- [mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesis](https://mlsanity.com/paper/1909.11504/) (2019)
- [MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generation](https://mlsanity.com/paper/2304.04106/) (2023)
- [Echo from noise: synthetic ultrasound image generation using diffusion models for real image segmentation](https://mlsanity.com/paper/2305.05424/) (2023)
- [Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis](https://mlsanity.com/paper/2303.11224/) (2023)
- [Synthetic CT Generation from MRI using 3D Transformer-based Denoising Diffusion Model](https://mlsanity.com/paper/2305.19467/) (2023)
- [EVA: Generating Longitudinal Electronic Health Records Using Conditional Variational Autoencoders](https://mlsanity.com/paper/2012.10020/) (2020)
- [Natural Language Generation for Electronic Health Records](https://mlsanity.com/paper/1806.01353/) (2018)

## Built on by (most cited first, 24 of 24)

- [Beyond Generative AI: World Models for Clinical Prediction, Counterfactuals, and Planning](https://mlsanity.com/paper/2511.16333/) (2025)
- [Longitudinal Flow Matching for Trajectory Modeling](https://mlsanity.com/paper/2510.03569/) (2025)
- [SimSUM: Simulated Benchmark with Structured and Unstructured Medical Records](https://mlsanity.com/paper/2409.08936/) (2024)
- [Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling](https://mlsanity.com/paper/2502.09688/) (2025)
- [Radiology Report Conditional 3D CT Generation with Multi Encoder Latent diffusion Model](https://mlsanity.com/paper/2509.14780/) (2025)
- [SoK: Privacy-aware LLM in Healthcare: Threat Model, Privacy Techniques, Challenges and Recommendations](https://mlsanity.com/paper/2601.10004/) (2026)
- [High-precision medical speech recognition through synthetic data and semantic correction: UNITED-MEDASR](https://mlsanity.com/paper/2412.00055/) (2024)
- [FundusGAN: A Hierarchical Feature-Aware Generative Framework for High-Fidelity Fundus Image Generation](https://mlsanity.com/paper/2503.17831/) (2025)
- [Interpretable Similarity of Synthetic Image Utility](https://mlsanity.com/paper/2512.17080/) (2025)
- [MRI-to-CT synthesis using drifting models](https://mlsanity.com/paper/2603.28498/) (2026)
- [SRA-Seg: Synthetic to Real Alignment for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2602.02944/) (2026)
- [MeDUET: Disentangled Unified Pretraining for 3D Medical Image Synthesis and Analysis](https://mlsanity.com/paper/2602.17901/) (2026)
- [Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesis](https://mlsanity.com/paper/2602.18400/) (2026)
- [Improving Diagnostic Accuracy for Oral Cancer with inpainting Synthesis Lesions Generated Using Diffusion Models](https://mlsanity.com/paper/2508.06151/) (2025)
- [Enabling Granular Subgroup Level Model Evaluations by Generating Synthetic Medical Time Series](https://mlsanity.com/paper/2510.19728/) (2025)
- [Generative Artificial Intelligence in Bioinformatics: A Systematic Review of Models, Applications, and Methodological Advances](https://mlsanity.com/paper/2511.03354/) (2025)
- [Interpretability and Individuality in Knee MRI: Patient-Specific Radiomic Fingerprint with Reconstructed Healthy Personas](https://mlsanity.com/paper/2601.08604/) (2026)
- [Evaluation metrics for temporal preservation in synthetic longitudinal patient data](https://mlsanity.com/paper/2602.10643/) (2026)
- [Discriminative Span as a Predictor of Synthetic Data Utility via Classifier Reconstruction](https://mlsanity.com/paper/2605.09697/) (2026)
- [LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators](https://mlsanity.com/paper/2605.19060/) (2026)
- [Structural MRI Synthesis for Alzheimer's Disease via Conditional Diffusion on Anatomical Masks](https://mlsanity.com/paper/2606.18354/) (2026)
- [Feature-Space Guided Diffusion for Realistic Ultrasound Image Synthesis](https://mlsanity.com/paper/2607.11655/) (2026)
- [A Foundational EDM2-Based Generative Model for High-Resolution Synthetic Fetal Ultrasound Imaging from Open Datasets](https://mlsanity.com/paper/2608.05471/) (2026)
- [Clinical Communication Processing with Models Trained on LLM-Generated Synthetic Data: A Structured Survey and Novel Application Case Studies](https://mlsanity.com/paper/2608.05993/) (2026)

## Abstract (arXiv)

This paper presents a comprehensive systematic review of generative models (GANs, VAEs, DMs, and LLMs) used to synthesize various medical data types, including imaging (dermoscopic, mammographic, ultrasound, CT, MRI, and X-ray), text, time-series, and tabular data (EHR). Unlike previous narrowly focused reviews, our study encompasses a broad array of medical data modalities and explores various generative models. Our search strategy queries databases such as Scopus, PubMed, and ArXiv, focusing on recent works from January 2021 to November 2023, excluding reviews and perspectives. This period emphasizes recent advancements beyond GANs, which have been extensively covered previously. The survey reveals insights from three key aspects: (1) Synthesis applications and purpose of synthesis, (2) generation techniques, and (3) evaluation methods. It highlights clinically valid synthesis applications, demonstrating the potential of synthetic data to tackle diverse clinical requirements. While conditional models incorporating class labels, segmentation masks and image translations are prevalent, there is a gap in utilizing prior clinical knowledge and patient-specific context, suggesting a need for more personalized synthesis approaches and emphasizing the importance of tailoring generative approaches to the unique characteristics of medical data. Additionally, there is a significant gap in using synthetic data beyond augmentation, such as for validation and evaluation of downstream medical AI models. The survey uncovers that the lack of standardized evaluation methodologies tailored to medical images is a barrier to clinical application, underscoring the need for in-depth evaluation approaches, benchmarking, and comparative studies to promote openness and collaboration.