# Brain tumor segmentation using synthetic MR images -- A comparison of GANs and diffusion models

arXiv: https://arxiv.org/abs/2306.02986 · HTML: https://mlsanity.com/paper/2306.02986/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 29 of its 57 references that are in the map.
- First posted on arXiv 5 Jun 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification](https://mlsanity.com/paper/1803.01229/) (2018)
- [A Note on the Inception Score](https://mlsanity.com/paper/1801.01973/) (2018)
- [Pros and Cons of GAN Evaluation Measures](https://mlsanity.com/paper/1802.03446/) (2018)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks](https://mlsanity.com/paper/1810.10863/) (2018)
- [SinGAN-Seg: Synthetic training data generation for medical image segmentation](https://mlsanity.com/paper/2107.00471/) (2021)
- [Can segmentation models be trained with fully synthetically generated data?](https://mlsanity.com/paper/2209.08256/) (2022)
- [Synthetic Medical Images from Dual Generative Adversarial Networks](https://mlsanity.com/paper/1709.01872/) (2017)
- [Beware of diffusion models for synthesizing medical images -- A comparison with GANs in terms of memorizing brain MRI and chest x-ray images](https://mlsanity.com/paper/2305.07644/) (2023)
- [Synthesizing brain tumor images and annotations by combining progressive growing GAN and SPADE](https://mlsanity.com/paper/2009.05946/) (2020)
- [Ensembles of GANs for synthetic training data generation](https://mlsanity.com/paper/2104.11797/) (2021)
- [DiffuseExpand: Expanding dataset for 2D medical image segmentation using diffusion models](https://mlsanity.com/paper/2304.13416/) (2023)
- [Does an ensemble of GANs lead to better performance when training segmentation networks with synthetic images?](https://mlsanity.com/paper/2211.04086/) (2022)

## Built on by (most cited first, 19 of 19)

- [BRISC: Annotated Dataset for Brain Tumor Segmentation and Classification](https://mlsanity.com/paper/2506.14318/) (2025)
- [Replication in Visual Diffusion Models: A Survey and Outlook](https://mlsanity.com/paper/2408.00001/) (2024)
- [Diffusion-empowered AutoPrompt MedSAM](https://mlsanity.com/paper/2502.06817/) (2025)
- [A Survey of Emerging Applications of Diffusion Probabilistic Models in MRI](https://mlsanity.com/paper/2311.11383/) (2023)
- [Assessing the capacity of a denoising diffusion probabilistic model to reproduce spatial context](https://mlsanity.com/paper/2309.10817/) (2023)
- [Synthetic Tumor Manipulation: With Radiomics Features](https://mlsanity.com/paper/2311.02586/) (2023)
- [Paired Diffusion: Generation of related, synthetic PET-CT-Segmentation scans using Linked Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2403.17734/) (2024)
- [Can Diffusion Models Bridge the Domain Gap in Cardiac MR Imaging?](https://mlsanity.com/paper/2508.06327/) (2025)
- [Diffusion-Driven Generation of Minimally Preprocessed Brain MRI](https://mlsanity.com/paper/2510.26834/) (2025)
- [TumorFlow: Physics-Guided Longitudinal MRI Synthesis of Glioblastoma Growth](https://mlsanity.com/paper/2603.04058/) (2026)
- [Leveraging Modified Ex Situ Tomography Data for Segmentation of In Situ Synchrotron X-Ray Computed Tomography](https://mlsanity.com/paper/2504.19200/) (2025)
- [Efficacy of Image Similarity as a Metric for Augmenting Small Dataset Retinal Image Segmentation](https://mlsanity.com/paper/2507.04862/) (2025)
- [Do Synthetic Brain MRIs Reliably Improve Tumour Classification? A StyleGAN2-ADA Class-Plane Augmentation Study on BRISC 2025](https://mlsanity.com/paper/2605.23094/) (2026)
- [VITON-DRR: Details Retention Virtual Try-on via Non-rigid Registration](https://mlsanity.com/paper/2505.23439/) (2025)
- [Proportional Sensitivity in Generative Adversarial Network (GAN)-Augmented Brain Tumor Classification Using Convolutional Neural Network](https://mlsanity.com/paper/2506.17165/) (2025)
- [Assessment of Using Synthetic Data in Brain Tumor Segmentation](https://mlsanity.com/paper/2508.11922/) (2025)
- [ALL-PET: A Low-resource and Low-shot PET Foundation Model in Projection Domain](https://mlsanity.com/paper/2509.09130/) (2025)
- [Phase-map synthesis from magnitude-only MR images using conditional score-based diffusion models with application in training of accelerated MRI reconstruction models](https://mlsanity.com/paper/2605.01185/) (2026)
- [FairGen: Preference-Aligned Diffusion for Demographically Equitable Medical Image Synthesis](https://mlsanity.com/paper/2606.14727/) (2026)

## Abstract (arXiv)

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1-3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80% - 90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub