# Augmenting medical image classifiers with synthetic data from latent diffusion models arXiv: https://arxiv.org/abs/2308.12453 ยท HTML: https://mlsanity.com/paper/2308.12453/ ## Key facts - Cited by 22 papers in the map (data through 14 Oct 2026). - Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. - Builds on 25 of its 45 references that are in the map. - First posted on arXiv 23 Aug 2023. ## Placed in - Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/) ## Builds on (most cited first, 25 of 25) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022) - [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019) - [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021) - [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023) - [Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset](https://mlsanity.com/paper/2104.09957/) (2021) - [fastai: A Layered API for Deep Learning](https://mlsanity.com/paper/2002.04688/) (2020) - [Disparities in Dermatology AI: Assessments Using Diverse Clinical Images](https://mlsanity.com/paper/2111.08006/) (2021) - [A deep learning system for differential diagnosis of skin diseases](https://mlsanity.com/paper/1909.05382/) (2019) - [Generative models improve fairness of medical classifiers under distribution shifts](https://mlsanity.com/paper/2304.09218/) (2023) - [Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains](https://mlsanity.com/paper/2210.04133/) (2022) - [Differentially Private Diffusion Models Generate Useful Synthetic Images](https://mlsanity.com/paper/2302.13861/) (2023) - [Diffusion-based Data Augmentation for Skin Disease Classification: Impact Across Original Medical Datasets to Fully Synthetic Images](https://mlsanity.com/paper/2301.04802/) (2023) - [Generation of Anonymous Chest Radiographs Using Latent Diffusion Models for Training Thoracic Abnormality Classification Systems](https://mlsanity.com/paper/2211.01323/) (2022) - [Overcoming Barriers to Data Sharing with Medical Image Generation: A Comprehensive Evaluation](https://mlsanity.com/paper/2012.03769/) (2020) - [Selective Synthetic Augmentation with HistoGAN for Improved Histopathology Image Classification](https://mlsanity.com/paper/2111.06399/) (2021) - [Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithm](https://mlsanity.com/paper/2207.02942/) (2022) - [DermGAN: Synthetic Generation of Clinical Skin Images with Pathology](https://mlsanity.com/paper/1911.08716/) (2019) - [Improving dermatology classifiers across populations using images generated by large diffusion models](https://mlsanity.com/paper/2211.13352/) (2022) - [Reducing bias and increasing utility by federated generative modeling of medical images using a centralized adversary](https://mlsanity.com/paper/2101.07235/) (2021) - [Intrinsic Self-Supervision for Data Quality Audits](https://mlsanity.com/paper/2305.17048/) (2023) ## Built on by (most cited first, 22 of 22) - [WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis](https://mlsanity.com/paper/2402.19043/) (2024) - [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024) - [From Majority to Minority: A Diffusion-based Augmentation for Underrepresented Groups in Skin Lesion Analysis](https://mlsanity.com/paper/2406.18375/) (2024) - [Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback](https://mlsanity.com/paper/2506.12323/) (2025) - [Ultrasound Image Generation using Latent Diffusion Models](https://mlsanity.com/paper/2502.08580/) (2025) - [A Generalist Model for Diverse Text-Guided Medical Image Synthesis](https://mlsanity.com/paper/2405.09806/) (2024) - [S-SYNTH: Knowledge-Based, Synthetic Generation of Skin Images](https://mlsanity.com/paper/2408.00191/) (2024) - [Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Survey](https://mlsanity.com/paper/2409.18214/) (2024) - [Ctrl-GenAug: Controllable Generative Augmentation for Medical Sequence Classification](https://mlsanity.com/paper/2409.17091/) (2024) - [LesionGen: A Concept-Guided Diffusion Model for Dermatology Image Synthesis](https://mlsanity.com/paper/2507.23001/) (2025) - [Knowledge-driven AI-generated data for accurate and interpretable breast ultrasound diagnoses](https://mlsanity.com/paper/2407.16634/) (2024) - [Bayesian Power Steering: An Effective Approach for Domain Adaptation of Diffusion Models](https://mlsanity.com/paper/2406.03683/) (2024) - [GAUDA: Generative Adaptive Uncertainty-guided Diffusion-based Augmentation for Surgical Segmentation](https://mlsanity.com/paper/2501.10819/) (2025) - [Evaluating Utility of Memory Efficient Medical Image Generation: A Study on Lung Nodule Segmentation](https://mlsanity.com/paper/2410.12542/) (2024) - [Automated Measurement of Eczema Severity with Self-Supervised Learning](https://mlsanity.com/paper/2504.15193/) (2025) - [SurrogateSHAP: Training-Free Contributor Attribution for Text-to-Image (T2I) Models](https://mlsanity.com/paper/2601.22276/) (2026) - [Few-Shot Synthetic Data Generation with Diffusion Models for Downstream Vision Tasks](https://mlsanity.com/paper/2605.11898/) (2026) - [Structural MRI Synthesis for Alzheimer's Disease via Conditional Diffusion on Anatomical Masks](https://mlsanity.com/paper/2606.18354/) (2026) - [Controllable Generation of Diverse Dermatological Imagery for Fair and Efficient Malignancy Classification](https://mlsanity.com/paper/2607.12987/) (2026) - [To do(x) or not to do(x): Medical Image Counterfactuals for Dataset Augmentation](https://mlsanity.com/paper/2609.14124/) (2026) - [Statistical attribute alignment for black-box generative AI via output post-processing](https://mlsanity.com/paper/2609.31607/) (2026) - [Generative AI-Based Data Augmentation for Oral Lesion Classification: The PhotoMOCI Dataset and Benchmark](https://mlsanity.com/paper/2609.35226/) (2026) ## Abstract (arXiv) While hundreds of artificial intelligence (AI) algorithms are now approved or cleared by the US Food and Drugs Administration (FDA), many studies have shown inconsistent generalization or latent bias, particularly for underrepresented populations. Some have proposed that generative AI could reduce the need for real data, but its utility in model development remains unclear. Skin disease serves as a useful case study in synthetic image generation due to the diversity of disease appearance, particularly across the protected attribute of skin tone. Here we show that latent diffusion models can scalably generate images of skin disease and that augmenting model training with these data improves performance in data-limited settings. These performance gains saturate at synthetic-to-real image ratios above 10:1 and are substantially smaller than the gains obtained from adding real images. As part of our analysis, we generate and analyze a new dataset of 458,920 synthetic images produced using several generation strategies. Our results suggest that synthetic data could serve as a force-multiplier for model development, but the collection of diverse real-world data remains the most important step to improve medical AI algorithms.