# Memory-Efficient 3D Denoising Diffusion Models for Medical Image Processing

arXiv: https://arxiv.org/abs/2303.15288 · HTML: https://mlsanity.com/paper/2303.15288/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 20 of its 27 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [SegDiff: Image Segmentation with Diffusion Probabilistic Models](https://mlsanity.com/paper/2112.00390/) (2021)
- [Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise](https://mlsanity.com/paper/2208.09392/) (2022)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model](https://mlsanity.com/paper/2211.00611/) (2022)
- [Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation](https://mlsanity.com/paper/2211.03364/) (2022)
- [Accelerating Diffusion Models via Pre-segmentation Diffusion Sampling for Medical Image Segmentation](https://mlsanity.com/paper/2210.17408/) (2022)

## Built on by (most cited first, 30 of 30)

- [WDM: 3D Wavelet Diffusion Models for High-Resolution Medical Image Synthesis](https://mlsanity.com/paper/2402.19043/) (2024)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [DiffBoost: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model](https://mlsanity.com/paper/2310.12868/) (2023)
- [Denoising Diffusion Models for 3D Healthy Brain Tissue Inpainting](https://mlsanity.com/paper/2403.14499/) (2024)
- [Denoising Diffusion Models for Inpainting of Healthy Brain Tissue](https://mlsanity.com/paper/2402.17307/) (2024)
- [2.5D Multi-view Averaging Diffusion Model for 3D Medical Image Translation: Application to Low-count PET Reconstruction with CT-less Attenuation Correction](https://mlsanity.com/paper/2406.08374/) (2024)
- [A Survey of Emerging Applications of Diffusion Probabilistic Models in MRI](https://mlsanity.com/paper/2311.11383/) (2023)
- [Pi-fusion: Physics-informed diffusion model for learning fluid dynamics](https://mlsanity.com/paper/2406.03711/) (2024)
- [SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models](https://mlsanity.com/paper/2412.02332/) (2024)
- [A Recycling Training Strategy for Medical Image Segmentation with Diffusion Denoising Models](https://mlsanity.com/paper/2308.16355/) (2023)
- [Brain Tumour Removing and Missing Modality Generation using 3D WDM](https://mlsanity.com/paper/2411.04630/) (2024)
- [LAND: Lung and Nodule Diffusion for 3D Chest CT Synthesis with Anatomical Guidance](https://mlsanity.com/paper/2510.18446/) (2025)
- [Denoising Diffusion Models for Anomaly Localization in Medical Images](https://mlsanity.com/paper/2410.23834/) (2024)
- [TextDiffSeg: Text-guided Latent Diffusion Model for 3d Medical Images Segmentation](https://mlsanity.com/paper/2504.11825/) (2025)
- [QID2: An Image-Conditioned Diffusion Model for Q-space Up-sampling of DWI Data](https://mlsanity.com/paper/2409.02309/) (2024)
- [Continuous and complete liver vessel segmentation with graph-attention guided diffusion](https://mlsanity.com/paper/2411.00617/) (2024)
- [Probabilistic Forecasting for Dynamical Systems with Missing or Imperfect Data](https://mlsanity.com/paper/2503.12273/) (2025)
- [RealDeal: Enhancing Realism and Details in Brain Image Generation via Image-to-Image Diffusion Models](https://mlsanity.com/paper/2507.18830/) (2025)
- [MotionDPS: Motion-Compensated 3D Brain MRI Reconstruction](https://mlsanity.com/paper/2605.22121/) (2026)
- [IgCONDA-PET: Weakly-Supervised PET Anomaly Detection using Implicitly-Guided Attention-Conditional Counterfactual Diffusion Modeling -- a Multi-Center, Multi-Cancer, and Multi-Tracer Study](https://mlsanity.com/paper/2405.00239/) (2024)
- [Evaluating Utility of Memory Efficient Medical Image Generation: A Study on Lung Nodule Segmentation](https://mlsanity.com/paper/2410.12542/) (2024)
- [Exploring Robustness of Cortical Morphometry in the presence of white matter lesions, using Diffusion Models for Lesion Filling](https://mlsanity.com/paper/2503.20571/) (2025)
- [Efficient and robust 3D blind harmonization for large domain gaps](https://mlsanity.com/paper/2505.00133/) (2025)
- [An Efficient 3D Latent Diffusion Model for T1-contrast Enhanced MRI Generation](https://mlsanity.com/paper/2509.24194/) (2025)
- [AI-Driven Radiology Report Generation for Traumatic Brain Injuries](https://mlsanity.com/paper/2510.08498/) (2025)
- [Kidney Cancer Detection Using 3D-Based Latent Diffusion Models](https://mlsanity.com/paper/2601.05852/) (2026)
- [Structure-Adaptive Sparse Diffusion in Voxel Space for 3D Medical Image Enhancement](https://mlsanity.com/paper/2604.17773/) (2026)
- [LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators](https://mlsanity.com/paper/2605.19060/) (2026)
- [Spatio-Temporal Mixture-of-Modality-Experts Diffusion for Quantitative DCE-MRI Synthesis from Incomplete MR Sequences](https://mlsanity.com/paper/2606.25535/) (2026)
- [Renormalization Group Flow Matching for Scalable Local Generative Modeling](https://mlsanity.com/paper/2608.23696/) (2026)

## Abstract (arXiv)

Denoising diffusion models have recently achieved state-of-the-art performance in many image-generation tasks. They do, however, require a large amount of computational resources. This limits their application to medical tasks, where we often deal with large 3D volumes, like high-resolution three-dimensional data. In this work, we present a number of different ways to reduce the resource consumption for 3D diffusion models and apply them to a dataset of 3D images. The main contribution of this paper is the memory-efficient patch-based diffusion model textitPatchDDM, which can be applied to the total volume during inference while the training is performed only on patches. While the proposed diffusion model can be applied to any image generation tasks, we evaluate the method on the tumor segmentation task of the BraTS2020 dataset and demonstrate that we can generate meaningful three-dimensional segmentations.