# Denoising Diffusion Models for 3D Healthy Brain Tissue Inpainting

arXiv: https://arxiv.org/abs/2403.14499 · HTML: https://mlsanity.com/paper/2403.14499/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [Wavelet Diffusion Models are fast and scalable Image Generators](https://mlsanity.com/paper/2211.16152/) (2022)
- [Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE](https://mlsanity.com/paper/2103.10022/) (2021)
- [Aggregated Contextual Transformations for High-Resolution Image Inpainting](https://mlsanity.com/paper/2104.01431/) (2021)
- [MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation](https://mlsanity.com/paper/2110.01406/) (2021)
- [WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis](https://mlsanity.com/paper/2402.19043/) (2024)
- [Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation](https://mlsanity.com/paper/2211.03364/) (2022)
- [Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis](https://mlsanity.com/paper/2307.10094/) (2023)
- [Memory-Efficient 3D Denoising Diffusion Models for Medical Image Processing](https://mlsanity.com/paper/2303.15288/) (2023)
- [Point Cloud Diffusion Models for Automatic Implant Generation](https://mlsanity.com/paper/2303.08061/) (2023)
- [Diffusion Models for Contrast Harmonization of Magnetic Resonance Images](https://mlsanity.com/paper/2303.08189/) (2023)
- [Denoising Diffusion Models for Inpainting of Healthy Brain Tissue](https://mlsanity.com/paper/2402.17307/) (2024)

## Built on by (most cited first, 22 of 22)

- [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024)
- [Regression is all you need for medical image translation](https://mlsanity.com/paper/2505.02048/) (2025)
- [fastWDM3D: Fast and Accurate 3D Healthy Tissue Inpainting](https://mlsanity.com/paper/2507.13146/) (2025)
- [Denoising Plane Wave Ultrasound Images Using Diffusion Probabilistic Models](https://mlsanity.com/paper/2408.10987/) (2024)
- [Conditional diffusion models for guided anomaly detection in brain images using fluid-driven anomaly randomization](https://mlsanity.com/paper/2506.10233/) (2025)
- [Brain Tumour Removing and Missing Modality Generation using 3D WDM](https://mlsanity.com/paper/2411.04630/) (2024)
- [Denoising Diffusion Models for Anomaly Localization in Medical Images](https://mlsanity.com/paper/2410.23834/) (2024)
- [DiffKAN-Inpainting: KAN-based Diffusion model for brain tumor inpainting](https://mlsanity.com/paper/2502.16771/) (2025)
- [High Volume Rate 3D Ultrasound Reconstruction with Diffusion Models](https://mlsanity.com/paper/2505.22090/) (2025)
- [RealDeal: Enhancing Realism and Details in Brain Image Generation via Image-to-Image Diffusion Models](https://mlsanity.com/paper/2507.18830/) (2025)
- [Towards MR-Based Trochleoplasty Planning](https://mlsanity.com/paper/2508.06076/) (2025)
- [Exploring Robustness of Cortical Morphometry in the presence of white matter lesions, using Diffusion Models for Lesion Filling](https://mlsanity.com/paper/2503.20571/) (2025)
- [3D Shape Generation: A Survey](https://mlsanity.com/paper/2506.22678/) (2025)
- [Deep generative priors for 3D brain analysis](https://mlsanity.com/paper/2510.15119/) (2025)
- [BrainNormalizer: Anatomy-Informed Pseudo-Healthy Brain Reconstruction from Tumor MRI via Edge-Guided ControlNet](https://mlsanity.com/paper/2511.12853/) (2025)
- [Kidney Cancer Detection Using 3D-Based Latent Diffusion Models](https://mlsanity.com/paper/2601.05852/) (2026)
- [POWDR: Pathology-preserving Outpainting with Wavelet Diffusion for 3D MRI](https://mlsanity.com/paper/2601.09044/) (2026)
- [Longitudinal Lesion Inpainting in Brain MRI via 3D Region Aware Diffusion](https://mlsanity.com/paper/2603.05693/) (2026)
- [Diffusion Models in Medical Image Inpainting: Challenges, Solution Taxonomy, and Future Directions](https://mlsanity.com/paper/2607.21904/) (2026)
- [Now You Have My Healthy Attention: A U-DiT for Brain-MRI Inpainting](https://mlsanity.com/paper/2607.27974/) (2026)
- [RARF: Region-Aware Rectified Flows for 3D Brain MRI Inpainting](https://mlsanity.com/paper/2609.03956/) (2026)
- [CATCH: Counterfactual Anatomical Tissue Inpainting with Conditional Haar Diffusion](https://mlsanity.com/paper/2609.29591/) (2026)

## Abstract (arXiv)

Monitoring diseases that affect the brain's structural integrity requires automated analysis of magnetic resonance (MR) images, e.g., for the evaluation of volumetric changes. However, many of the evaluation tools are optimized for analyzing healthy tissue. To enable the evaluation of scans containing pathological tissue, it is therefore required to restore healthy tissue in the pathological areas. In this work, we explore and extend denoising diffusion models for consistent inpainting of healthy 3D brain tissue. We modify state-of-the-art 2D, pseudo-3D, and 3D methods working in the image space, as well as 3D latent and 3D wavelet diffusion models, and train them to synthesize healthy brain tissue. Our evaluation shows that the pseudo-3D model performs best regarding the structural-similarity index, peak signal-to-noise ratio, and mean squared error. To emphasize the clinical relevance, we fine-tune this model on data containing synthetic MS lesions and evaluate it on a downstream brain tissue segmentation task, whereby it outperforms the established FMRIB Software Library (FSL) lesion-filling method.