# SwinTF3D: A Lightweight Multimodal Fusion Approach for Text-Guided 3D Medical Image Segmentation

arXiv: https://arxiv.org/abs/2512.22878 · HTML: https://mlsanity.com/paper/2512.22878/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 28 Dec 2025.

## Builds on (most cited first, 25 of 25)

- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP](https://mlsanity.com/paper/2210.04150/) (2022)
- [nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation](https://mlsanity.com/paper/2404.09556/) (2024)
- [nnFormer: Interleaved Transformer for Volumetric Segmentation](https://mlsanity.com/paper/2109.03201/) (2021)
- [Medical Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2009.13120/) (2020)
- [MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation](https://mlsanity.com/paper/2303.09975/) (2023)
- [UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation](https://mlsanity.com/paper/2212.04497/) (2022)
- [LViT: Language meets Vision Transformer in Medical Image Segmentation](https://mlsanity.com/paper/2206.14718/) (2022)
- [MISSFormer: An Effective Medical Image Segmentation Transformer](https://mlsanity.com/paper/2109.07162/) (2021)
- [Large-Vocabulary Segmentation for Medical Images with Text Prompts](https://mlsanity.com/paper/2312.17183/) (2023)
- [Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis](https://mlsanity.com/paper/2503.20047/) (2025)
- [BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP](https://mlsanity.com/paper/2506.10896/) (2025)
- [MIS-FM: 3D Medical Image Segmentation using Foundation Models Pretrained on a Large-Scale Unannotated Dataset](https://mlsanity.com/paper/2306.16925/) (2023)
- [AutoProSAM: Automated Prompting SAM for 3D Multi-Organ Segmentation](https://mlsanity.com/paper/2308.14936/) (2023)
- [Promise:Prompt-driven 3D Medical Image Segmentation Using Pretrained Image Foundation Models](https://mlsanity.com/paper/2310.19721/) (2023)
- [E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation](https://mlsanity.com/paper/2312.04727/) (2023)
- [EM-Net: Efficient Channel and Frequency Learning with Mamba for 3D Medical Image Segmentation](https://mlsanity.com/paper/2409.17675/) (2024)
- [Med3DInsight: Enhancing 3D Medical Image Understanding with 2D Multi-Modal Large Language Models](https://mlsanity.com/paper/2403.05141/) (2024)
- [UKBOB: One Billion MRI Labeled Masks for Generalizable 3D Medical Image Segmentation](https://mlsanity.com/paper/2504.06908/) (2025)
- [Medverse: A Universal Model for Full-Resolution 3D Medical Image Segmentation, Transformation and Enhancement](https://mlsanity.com/paper/2509.09232/) (2025)
- [SegResMamba: An Efficient Architecture for 3D Medical Image Segmentation](https://mlsanity.com/paper/2503.07766/) (2025)
- [TextDiffSeg: Text-guided Latent Diffusion Model for 3d Medical Images Segmentation](https://mlsanity.com/paper/2504.11825/) (2025)
- [Dynamic Prompt Generation for Interactive 3D Medical Image Segmentation Training](https://mlsanity.com/paper/2510.03189/) (2025)

## Abstract (arXiv)

The recent integration of artificial intelligence into medical imaging has driven remarkable advances in automated organ segmentation. However, most existing 3D segmentation frameworks rely exclusively on visual learning from large annotated datasets restricting their adaptability to new domains and clinical tasks. The lack of semantic understanding in these models makes them ineffective in addressing flexible, user-defined segmentation objectives. To overcome these limitations, we propose SwinTF3D, a lightweight multimodal fusion approach that unifies visual and linguistic representations for text-guided 3D medical image segmentation. The model employs a transformer-based visual encoder to extract volumetric features and integrates them with a compact text encoder via an efficient fusion mechanism. This design allows the system to understand natural-language prompts and correctly align semantic cues with their corresponding spatial structures in medical volumes, while producing accurate, context-aware segmentation results with low computational overhead. Extensive experiments on the BTCV dataset demonstrate that SwinTF3D achieves competitive Dice and IoU scores across multiple organs, despite its compact architecture. The model generalizes well to unseen data and offers significant efficiency gains compared to conventional transformer-based segmentation networks. Bridging visual perception with linguistic understanding, SwinTF3D establishes a practical and interpretable paradigm for interactive, text-driven 3D medical image segmentation, opening perspectives for more adaptive and resource-efficient solutions in clinical imaging.