SwinTF3D: A Lightweight Multimodal Fusion Approach for Text-Guided 3D Medical Image Segmentation
Not on the map. Why?
Abstract (arXiv)
The recent integration of artificial intelligence into medical imaging has driven remarkable advances in automated organ segmentation. However, most existing 3D segmentation frameworks rely exclusively on visual learning from large annotated datasets restricting their adaptability to new domains and clinical tasks. The lack of semantic understanding in these models makes them ineffective in addressing flexible, user-defined segmentation objectives. To overcome these limitations, we propose SwinTF3D, a lightweight multimodal fusion approach that unifies visual and linguistic representations for text-guided 3D medical image segmentation. The model employs a transformer-based visual encoder to extract volumetric features and integrates them with a compact text encoder via an efficient fusion mechanism. This design allows the system to understand natural-language prompts and correctly align semantic cues with their corresponding spatial structures in medical volumes, while producing accurate, context-aware segmentation results with low computational overhead. Extensive experiments on the BTCV dataset demonstrate that SwinTF3D achieves competitive Dice and IoU scores across multiple organs, despite its compact architecture. The model generalizes well to unseen data and offers significant efficiency gains compared to conventional transformer-based segmentation networks. Bridging visual perception with linguistic understanding, SwinTF3D establishes a practical and interpretable paradigm for interactive, text-driven 3D medical image segmentation, opening perspectives for more adaptive and resource-efficient solutions in clinical imaging.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 28 Dec 2025.
Builds on
25 of 33 references.
References · 25
- 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
- 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
- 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
- 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
- 2024nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentationcited by 240
- 2021nnFormer: Interleaved Transformer for Volumetric Segmentationcited by 180
- 2020Medical Image Segmentation Using Deep Learning: A Surveycited by 164
- 2023MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentationcited by 162
- 2022UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentationcited by 112
- 2022LViT: Language meets Vision Transformer in Medical Image Segmentationcited by 111
- 2021MISSFormer: An Effective Medical Image Segmentation Transformercited by 103
- 2023Large-Vocabulary Segmentation for Medical Images with Text Promptscited by 62
13 more
- 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysiscited by 46
- 2025BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLPcited by 28
- 2023MIS-FM: 3D Medical Image Segmentation using Foundation Models Pretrained on a Large-Scale Unannotated Datasetcited by 26
- 2023AutoProSAM: Automated Prompting SAM for 3D Multi-Organ Segmentationcited by 19
- 2023Promise:Prompt-driven 3D Medical Image Segmentation Using Pretrained Image Foundation Modelscited by 18
- 2023E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentationcited by 16
- 2024EM-Net: Efficient Channel and Frequency Learning with Mamba for 3D Medical Image Segmentationcited by 12
- 2024Med3DInsight: Enhancing 3D Medical Image Understanding with 2D Multi-Modal Large Language Modelscited by 5
- 2025UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentationcited by 4
- 2025Medverse: A Universal Model for Full-Resolution 3D Medical Image Segmentation, Transformation and Enhancementcited by 3
- 2025SegResMamba: An Efficient Architecture for 3D Medical Image Segmentationcited by 2
- 2025TextDiffSeg: Text-guided Latent Diffusion Model for 3d Medical Images Segmentationcited by 2
- 2025Dynamic Prompt Generation for Interactive 3D Medical Image Segmentation Trainingcited by 2
Built on by
Not cited yet.