# Synthetic Data as Validation arXiv: https://arxiv.org/abs/2310.16052 ยท HTML: https://mlsanity.com/paper/2310.16052/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 23 of its 57 references that are in the map. - First posted on arXiv 24 Oct 2023. ## Builds on (most cited first, 23 of 23) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016) - [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016) - [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018) - [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019) - [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019) - [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023) - [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023) - [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023) - [Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach](https://mlsanity.com/paper/1812.05040/) (2018) - [Quant GANs: Deep Generation of Financial Time Series](https://mlsanity.com/paper/1907.06673/) (2019) - [Learning from Synthetic Animals](https://mlsanity.com/paper/1912.08265/) (2019) - [Label-Free Segmentation of COVID-19 Lesions in Lung CT](https://mlsanity.com/paper/2009.06456/) (2020) - [Label-Assemble: Leveraging Multiple Datasets with Partial Labels](https://mlsanity.com/paper/2109.12265/) (2021) - [Early Detection and Localization of Pancreatic Cancer by Label-Free Tumor Synthesis](https://mlsanity.com/paper/2308.03008/) (2023) - [Boomerang: Local sampling on image manifolds using diffusion models](https://mlsanity.com/paper/2210.12100/) (2022) - [Synthetic Tumors Make AI Segment Tumors Better](https://mlsanity.com/paper/2210.14845/) (2022) - [Image retrieval outperforms diffusion models on data augmentation](https://mlsanity.com/paper/2304.10253/) (2023) - [PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation](https://mlsanity.com/paper/2303.07337/) (2023) - [Less is More: Unsupervised Mask-guided Annotated CT Image Synthesis with Minimum Manual Segmentations](https://mlsanity.com/paper/2303.12747/) (2023) - [How Good Are Synthetic Medical Images? An Empirical Study with Lung Ultrasound](https://mlsanity.com/paper/2310.03608/) (2023) ## Built on by (most cited first, 10 of 10) - [Towards Generalizable Tumor Synthesis](https://mlsanity.com/paper/2402.19470/) (2024) - [AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarking](https://mlsanity.com/paper/2407.16697/) (2024) - [Endora: Video Generation Models as Endoscopy Simulators](https://mlsanity.com/paper/2403.11050/) (2024) - [From Pixel to Cancer: Cellular Automata in Computed Tomography](https://mlsanity.com/paper/2403.06459/) (2024) - [Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data](https://mlsanity.com/paper/2510.14831/) (2025) - [Analyzing Tumors by Synthesis](https://mlsanity.com/paper/2409.06035/) (2024) - [Expectation-Maximization as the Engine of Scalable Medical Intelligence](https://mlsanity.com/paper/2501.03410/) (2025) - [See More, Change Less: Anatomy-Aware Diffusion for Contrast Enhancement](https://mlsanity.com/paper/2512.07251/) (2025) - [Bi-modality medical images synthesis by a bi-directional discrete process matching method](https://mlsanity.com/paper/2409.03977/) (2024) - [Novel Methods for Catheter and Guidewire Segmentation in X-ray Fluoroscopy under a Federated Learning Setting](https://mlsanity.com/paper/2609.06876/) (2026) ## Abstract (arXiv) This study leverages synthetic data as a validation set to reduce overfitting and ease the selection of the best model in AI development. While synthetic data have been used for augmenting the training set, we find that synthetic data can also significantly diversify the validation set, offering marked advantages in domains like healthcare, where data are typically limited, sensitive, and from out-domain sources (i.e., hospitals). In this study, we illustrate the effectiveness of synthetic data for early cancer detection in computed tomography (CT) volumes, where synthetic tumors are generated and superimposed onto healthy organs, thereby creating an extensive dataset for rigorous validation. Using synthetic data as validation can improve AI robustness in both in-domain and out-domain test sets. Furthermore, we establish a new continual learning framework that continuously trains AI models on a stream of out-domain data with synthetic tumors. The AI model trained and validated in dynamically expanding synthetic data can consistently outperform models trained and validated exclusively on real-world data. Specifically, the DSC score for liver tumor segmentation improves from 26.7% (95% CI: 22.6%-30.9%) to 34.5% (30.8%-38.2%) when evaluated on an in-domain dataset and from 31.1% (26.0%-36.2%) to 35.4% (32.1%-38.7%) on an out-domain dataset. Importantly, the performance gain is particularly significant in identifying very tiny liver tumors (radius < 5mm) in CT volumes, with Sensitivity improving from 33.1% to 55.4% on an in-domain dataset and 33.9% to 52.3% on an out-domain dataset, justifying the efficacy in early detection of cancer. The application of synthetic data, from both training and validation perspectives, underlines a promising avenue to enhance AI robustness when dealing with data from varying domains.