# Diff-UNet: A Diffusion Embedded Network for Volumetric Segmentation

arXiv: https://arxiv.org/abs/2303.10326 · HTML: https://mlsanity.com/paper/2303.10326/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 21 of its 26 references that are in the map.
- First posted on arXiv 18 Mar 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model](https://mlsanity.com/paper/2211.00611/) (2022)

## Built on by (most cited first, 30 of 41)

- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation](https://mlsanity.com/paper/2401.13560/) (2024)
- [A Comprehensive Survey of Foundation Models in Medicine](https://mlsanity.com/paper/2406.10729/) (2024)
- [Towards Generic Semi-Supervised Framework for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2310.11320/) (2023)
- [Fast-DDPM: Fast Denoising Diffusion Probabilistic Models for Medical Image-to-Image Generation](https://mlsanity.com/paper/2405.14802/) (2024)
- [Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?](https://mlsanity.com/paper/2411.03670/) (2024)
- [Cross-conditioned Diffusion Model for Medical Image to Image Translation](https://mlsanity.com/paper/2409.08500/) (2024)
- [QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generation](https://mlsanity.com/paper/2503.06545/) (2025)
- [Stable Diffusion Segmentation for Biomedical Images with Single-step Reverse Process](https://mlsanity.com/paper/2406.18361/) (2024)
- [DiffRect: Latent Diffusion Label Rectification for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2407.09918/) (2024)
- [EM-Net: Efficient Channel and Frequency Learning with Mamba for 3D Medical Image Segmentation](https://mlsanity.com/paper/2409.17675/) (2024)
- [Advancing UWF-SLO Vessel Segmentation with Source-Free Active Domain Adaptation and a Novel Multi-Center Dataset](https://mlsanity.com/paper/2406.13645/) (2024)
- [A Survey of Emerging Applications of Diffusion Probabilistic Models in MRI](https://mlsanity.com/paper/2311.11383/) (2023)
- [Track Any Anomalous Object: A Granular Video Anomaly Detection Pipeline](https://mlsanity.com/paper/2506.05175/) (2025)
- [Timeline and Boundary Guided Diffusion Network for Video Shadow Detection](https://mlsanity.com/paper/2408.11785/) (2024)
- [Diff-VPS: Video Polyp Segmentation via a Multi-task Diffusion Network with Adversarial Temporal Reasoning](https://mlsanity.com/paper/2409.07238/) (2024)
- [Non-Invasive to Invasive: Enhancing FFA Synthesis from CFP with a Benchmark Dataset and a Novel Network](https://mlsanity.com/paper/2410.14965/) (2024)
- [A Recycling Training Strategy for Medical Image Segmentation with Diffusion Denoising Models](https://mlsanity.com/paper/2308.16355/) (2023)
- [FDiff-Fusion:Denoising diffusion fusion network based on fuzzy learning for 3D medical image segmentation](https://mlsanity.com/paper/2408.02075/) (2024)
- [TextDiffSeg: Text-guided Latent Diffusion Model for 3d Medical Images Segmentation](https://mlsanity.com/paper/2504.11825/) (2025)
- [Discrepancy-based Diffusion Models for Lesion Detection in Brain MRI](https://mlsanity.com/paper/2405.04974/) (2024)
- [Continuous and complete liver vessel segmentation with graph-attention guided diffusion](https://mlsanity.com/paper/2411.00617/) (2024)
- [Simulating Refractive Distortions and Weather-Induced Artifacts for Resource-Constrained Autonomous Perception](https://mlsanity.com/paper/2507.05536/) (2025)
- [Multimodal Fusion at Three Tiers: Physics-Driven Data Generation and Vision-Language Guidance for Brain Tumor Segmentation](https://mlsanity.com/paper/2507.09966/) (2025)
- [LEAF: Latent Diffusion with Efficient Encoder Distillation for Aligned Features in Medical Image Segmentation](https://mlsanity.com/paper/2507.18214/) (2025)
- [Evo: Autoregressive-Diffusion Large Language Models with Evolving Balance](https://mlsanity.com/paper/2603.06617/) (2026)
- [Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation](https://mlsanity.com/paper/2307.12180/) (2023)
- [Introducing Shape Prior Module in Diffusion Model for Medical Image Segmentation](https://mlsanity.com/paper/2309.05929/) (2023)
- [ResEnsemble-DDPM: Residual Denoising Diffusion Probabilistic Models for Ensemble Learning](https://mlsanity.com/paper/2312.01682/) (2023)
- [Re-DiffiNet: Modeling discrepancies in tumor segmentation using diffusion models](https://mlsanity.com/paper/2402.07354/) (2024)

## Abstract (arXiv)

In recent years, Denoising Diffusion Models have demonstrated remarkable success in generating semantically valuable pixel-wise representations for image generative modeling. In this study, we propose a novel end-to-end framework, called Diff-UNet, for medical volumetric segmentation. Our approach integrates the diffusion model into a standard U-shaped architecture to extract semantic information from the input volume effectively, resulting in excellent pixel-level representations for medical volumetric segmentation. To enhance the robustness of the diffusion model's prediction results, we also introduce a Step-Uncertainty based Fusion (SUF) module during inference to combine the outputs of the diffusion models at each step. We evaluate our method on three datasets, including multimodal brain tumors in MRI, liver tumors, and multi-organ CT volumes, and demonstrate that Diff-UNet outperforms other state-of-the-art methods significantly. Our experimental results also indicate the universality and effectiveness of the proposed model. The proposed framework has the potential to facilitate the accurate diagnosis and treatment of medical conditions by enabling more precise segmentation of anatomical structures. The codes of Diff-UNet are available at https://github.com/ge-xing/Diff-UNet