# Limitations of Synthetic Data Generation in Specialized Data-Scarce Domains

arXiv: https://arxiv.org/abs/2608.13729 · HTML: https://mlsanity.com/paper/2608.13729/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 13 Aug 2026.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Domain Adaptation for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2102.09508/) (2021)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification](https://mlsanity.com/paper/1803.01229/) (2018)
- [Conditional Text Image Generation with Diffusion Models](https://mlsanity.com/paper/2306.10804/) (2023)
- [Diffusion-based Data Augmentation for Skin Disease Classification: Impact Across Original Medical Datasets to Fully Synthetic Images](https://mlsanity.com/paper/2301.04802/) (2023)
- [Generation of Anonymous Chest Radiographs Using Latent Diffusion Models for Training Thoracic Abnormality Classification Systems](https://mlsanity.com/paper/2211.01323/) (2022)
- [Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Images](https://mlsanity.com/paper/2405.20469/) (2024)
- [Synthetic Medical Images from Dual Generative Adversarial Networks](https://mlsanity.com/paper/1709.01872/) (2017)
- [Improving dermatology classifiers across populations using images generated by large diffusion models](https://mlsanity.com/paper/2211.13352/) (2022)
- [The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better](https://mlsanity.com/paper/2406.05184/) (2024)
- [A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation](https://mlsanity.com/paper/2303.09165/) (2023)
- [Causal Explanations for Image Classifiers](https://mlsanity.com/paper/2411.08875/) (2024)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [Impact of Data Duplication on Deep Neural Network-Based Image Classifiers: Robust vs. Standard Models](https://mlsanity.com/paper/2504.00638/) (2025)
- [Unsupervised Welding Defect Detection Using Audio And Video](https://mlsanity.com/paper/2409.02290/) (2024)
- [3D ReX: Causal Explanations in 3D Neuroimaging Classification](https://mlsanity.com/paper/2502.12181/) (2025)
- [A Multi-Robot Platform for Robotic Triage Combining Onboard Sensing and Foundation Models](https://mlsanity.com/paper/2512.08754/) (2025)

## Abstract (arXiv)

Advances in diffusion-based generative models have motivated the use of synthetic image generation to alleviate data scarcity in vision tasks. While this strategy has shown promise in natural image benchmarks such as ImageNet, its effectiveness in sparse, high-variance real-world domains remains unclear. In this work, we focus on domains where images differ substantially from common image datasets and additional data are expensive to obtain. Against non-generative data augmentation baselines, we evaluate the downstream classifier performance improvements yielded by two schools of generative sparse data extension: distribution modeling and sample perturbation. Across five trauma classification tasks using subject-wise train--validation splits, no generative approach consistently outperforms a strong non-generative baseline. Feature-space analysis reveals recurring failure modes: memorization or collapse, distributional drift, and generation of visually plausible but simplified canonical instances that are easier to classify than real data.