# Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis

arXiv: https://arxiv.org/abs/2305.18453 · HTML: https://mlsanity.com/paper/2305.18453/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 29 May 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 27 of 27)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019)
- [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks](https://mlsanity.com/paper/1703.05192/) (2017)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [GANs for Medical Image Analysis](https://mlsanity.com/paper/1809.06222/) (2018)
- [Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networks](https://mlsanity.com/paper/1908.02498/) (2019)
- [GANs for Medical Image Synthesis: An Empirical Study](https://mlsanity.com/paper/2105.05318/) (2021)
- [Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN](https://mlsanity.com/paper/2008.01910/) (2020)
- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [Can segmentation models be trained with fully synthetically generated data?](https://mlsanity.com/paper/2209.08256/) (2022)
- [3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images](https://mlsanity.com/paper/2107.09700/) (2021)
- [Overfitting of neural nets under class imbalance: Analysis and improvements for segmentation](https://mlsanity.com/paper/1907.10982/) (2019)
- [Red-GAN: Attacking class imbalance via conditioned generation. Yet another perspective on medical image synthesis for skin lesion dermoscopy and brain tumor MRI](https://mlsanity.com/paper/2004.10734/) (2020)
- [Deep Learning Based Segmentation of Various Brain Lesions for Radiosurgery](https://mlsanity.com/paper/2007.11784/) (2020)

## Built on by (most cited first, 30 of 86)

- [Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing](https://mlsanity.com/paper/2407.01521/) (2024)
- [Flow Matching for Medical Image Synthesis: Bridging the Gap Between Speed and Quality](https://mlsanity.com/paper/2503.00266/) (2025)
- [DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model](https://mlsanity.com/paper/2310.12868/) (2023)
- [Robust CLIP-Based Detector for Exposing Diffusion Model-Generated Images](https://mlsanity.com/paper/2404.12908/) (2024)
- [vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation](https://mlsanity.com/paper/2411.17386/) (2024)
- [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024)
- [MedSegFactory: Text-Guided Generation of Medical Image-Mask Pairs](https://mlsanity.com/paper/2504.06897/) (2025)
- [Polyp-DDPM: Diffusion-Based Semantic Polyp Synthesis for Enhanced Segmentation](https://mlsanity.com/paper/2402.04031/) (2024)
- [A Review on Generative AI For Text-To-Image and Image-To-Image Generation and Implications To Scientific Images](https://mlsanity.com/paper/2502.21151/) (2025)
- [Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translation](https://mlsanity.com/paper/2508.09177/) (2025)
- [Synthetic Data in Radiological Imaging: Current State and Future Outlook](https://mlsanity.com/paper/2407.01561/) (2024)
- [Deformation-Recovery Diffusion Model (DRDM): Instance Deformation for Image Manipulation and Synthesis](https://mlsanity.com/paper/2407.07295/) (2024)
- [MedDiff-FM: A Diffusion-based Foundation Model for Versatile Medical Image Applications](https://mlsanity.com/paper/2410.15432/) (2024)
- [Lung-DDPM: Semantic Layout-guided Diffusion Models for Thoracic CT Image Synthesis](https://mlsanity.com/paper/2502.15204/) (2025)
- [WASABI: A Metric for Evaluating Morphometric Plausibility of Synthetic Brain MRIs](https://mlsanity.com/paper/2504.21771/) (2025)
- [ReGuidance: A Simple Diffusion Wrapper for Boosting Sample Quality on Hard Inverse Problems](https://mlsanity.com/paper/2506.10955/) (2025)
- [Ctrl-GenAug: Controllable Generative Augmentation for Medical Sequence Classification](https://mlsanity.com/paper/2409.17091/) (2024)
- [FairSkin: Fair Diffusion for Skin Disease Image Generation](https://mlsanity.com/paper/2410.22551/) (2024)
- [Lung-DDPM+: Efficient Thoracic CT Image Synthesis using Diffusion Probabilistic Model](https://mlsanity.com/paper/2508.09327/) (2025)
- [Super-resolution of biomedical volumes with 2D supervision](https://mlsanity.com/paper/2404.09425/) (2024)
- [Volumetric Conditioning Module to Control Pretrained Diffusion Models for 3D Medical Images](https://mlsanity.com/paper/2410.21826/) (2024)
- [MRGen: Segmentation Data Engine for Underrepresented MRI Modalities](https://mlsanity.com/paper/2412.04106/) (2024)
- [MedLoRD: A Medical Low-Resource Diffusion Model for High-Resolution 3D CT Image Synthesis](https://mlsanity.com/paper/2503.13211/) (2025)
- [STeP: A Framework for Solving Scientific Video Inverse Problems with Spatiotemporal Diffusion Priors](https://mlsanity.com/paper/2504.07549/) (2025)
- [EquiReg: Equivariance Regularized Diffusion for Inverse Problems](https://mlsanity.com/paper/2505.22973/) (2025)
- [Label-Efficient Data Augmentation with Video Diffusion Models for Guidewire Segmentation in Cardiac Fluoroscopy](https://mlsanity.com/paper/2412.16050/) (2024)
- [seg2med: a bridge from artificial anatomy to multimodal medical images](https://mlsanity.com/paper/2504.09182/) (2025)
- [CLAIM: Clinically-Guided LGE Augmentation for Realistic and Diverse Myocardial Scar Synthesis and Segmentation](https://mlsanity.com/paper/2506.15549/) (2025)
- [Flow Matching for Conditional MRI-CT and CBCT-CT Image Synthesis](https://mlsanity.com/paper/2510.04823/) (2025)
- [Diffusion-Driven Generation of Minimally Preprocessed Brain MRI](https://mlsanity.com/paper/2510.26834/) (2025)

## Abstract (arXiv)

Artificial intelligence (AI) in healthcare, especially in medical imaging, faces challenges due to data scarcity and privacy concerns. Addressing these, we introduce Med-DDPM, a diffusion model designed for 3D semantic brain MRI synthesis. This model effectively tackles data scarcity and privacy issues by integrating semantic conditioning. This involves the channel-wise concatenation of a conditioning image to the model input, enabling control in image generation. Med-DDPM demonstrates superior stability and performance compared to existing 3D brain imaging synthesis methods. It generates diverse, anatomically coherent images with high visual fidelity. In terms of dice score accuracy in the tumor segmentation task, Med-DDPM achieves 0.6207, close to the 0.6531 accuracy of real images, and outperforms baseline models. Combined with real images, it further increases segmentation accuracy to 0.6675, showing the potential of our proposed method for data augmentation. This model represents the first use of a diffusion model in 3D semantic brain MRI synthesis, producing high-quality images. Its semantic conditioning feature also shows potential for image anonymization in biomedical imaging, addressing data and privacy issues. We provide the code and model weights for Med-DDPM on our GitHub repository (https://github.com/mobaidoctor/med-ddpm/) to support reproducibility.