# TextDiffSeg: Text-guided Latent Diffusion Model for 3d Medical Images Segmentation

arXiv: https://arxiv.org/abs/2504.11825 · HTML: https://mlsanity.com/paper/2504.11825/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 26 of its 31 references that are in the map.
- First posted on arXiv 16 Apr 2025.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [U-Net and its variants for medical image segmentation: theory and applications](https://mlsanity.com/paper/2011.01118/) (2020)
- [UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation](https://mlsanity.com/paper/2212.04497/) (2022)
- [Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2308.09388/) (2023)
- [The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT](https://mlsanity.com/paper/2307.01984/) (2023)
- [3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2209.15076/) (2022)
- [Diff-UNet: A Diffusion Embedded Network for Volumetric Segmentation](https://mlsanity.com/paper/2303.10326/) (2023)
- [Memory-Efficient 3D Denoising Diffusion Models for Medical Image Processing](https://mlsanity.com/paper/2303.15288/) (2023)
- [Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks](https://mlsanity.com/paper/1702.05970/) (2017)
- [A Survey of Diffusion Based Image Generation Models: Issues and Their Solutions](https://mlsanity.com/paper/2308.13142/) (2023)
- [Slice-Consistent 3D Volumetric Brain CT-to-MRI Translation with 2D Brownian Bridge Diffusion Model](https://mlsanity.com/paper/2407.05059/) (2024)
- [Enhancing Label-efficient Medical Image Segmentation with Text-guided Diffusion Models](https://mlsanity.com/paper/2407.05323/) (2024)
- [Latent Diffusion for Medical Image Segmentation: End to end learning for fast sampling and accuracy](https://mlsanity.com/paper/2407.12952/) (2024)
- [Unsupervised Discovery of 3D Hierarchical Structure with Generative Diffusion Features](https://mlsanity.com/paper/2305.00067/) (2023)
- [FDiff-Fusion:Denoising diffusion fusion network based on fuzzy learning for 3D medical image segmentation](https://mlsanity.com/paper/2408.02075/) (2024)
- [RefSAM3D: Adapting SAM with Cross-modal Reference for 3D Medical Image Segmentation](https://mlsanity.com/paper/2412.05605/) (2024)

## Built on by (most cited first, 2 of 2)

- [MedEBench: Diagnosing Reliability in Text-Guided Medical Image Editing](https://mlsanity.com/paper/2506.01921/) (2025)
- [SwinTF3D: A Lightweight Multimodal Fusion Approach for Text-Guided 3D Medical Image Segmentation](https://mlsanity.com/paper/2512.22878/) (2025)

## Abstract (arXiv)

Diffusion Probabilistic Models (DPMs) have demonstrated significant potential in 3D medical image segmentation tasks. However, their high computational cost and inability to fully capture global 3D contextual information limit their practical applications. To address these challenges, we propose a novel text-guided diffusion model framework, TextDiffSeg. This method leverages a conditional diffusion framework that integrates 3D volumetric data with natural language descriptions, enabling cross-modal embedding and establishing a shared semantic space between visual and textual modalities. By enhancing the model's ability to recognize complex anatomical structures, TextDiffSeg incorporates innovative label embedding techniques and cross-modal attention mechanisms, effectively reducing computational complexity while preserving global 3D contextual integrity. Experimental results demonstrate that TextDiffSeg consistently outperforms existing methods in segmentation tasks involving kidney and pancreas tumors, as well as multi-organ segmentation scenarios. Ablation studies further validate the effectiveness of key components, highlighting the synergistic interaction between text fusion, image feature extractor, and label encoder. TextDiffSeg provides an efficient and accurate solution for 3D medical image segmentation, showcasing its broad applicability in clinical diagnosis and treatment planning.