# Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation arXiv: https://arxiv.org/abs/2607.16327 ยท HTML: https://mlsanity.com/paper/2607.16327/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 32 of its 55 references that are in the map. - First posted on arXiv 15 Jul 2026. ## Builds on (most cited first, 30 of 32) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022) - [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016) - [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018) - [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018) - [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021) - [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020) - [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023) - [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019) - [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023) - [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022) - [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022) - [LViT: Language meets Vision Transformer in Medical Image Segmentation](https://mlsanity.com/paper/2206.14718/) (2022) - [MosMedData: Chest CT Scans With COVID-19 Related Findings Dataset](https://mlsanity.com/paper/2005.06465/) (2020) - [ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detection](https://mlsanity.com/paper/2308.07504/) (2023) - [OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Images](https://mlsanity.com/paper/2202.10185/) (2022) - [A Survey on Ensemble Learning under the Era of Deep Learning](https://mlsanity.com/paper/2101.08387/) (2021) - [Ariadne's Thread:Using Text Prompts to Improve Segmentation of Infected Areas from Chest X-ray images](https://mlsanity.com/paper/2307.03942/) (2023) - [Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models](https://mlsanity.com/paper/2308.07706/) (2023) - [Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentation](https://mlsanity.com/paper/2404.02845/) (2024) - [Task-driven Image Fusion with Learnable Fusion Loss](https://mlsanity.com/paper/2412.03240/) (2024) - [Cross-aware Early Fusion with Stage-divided Vision and Language Transformer Encoders for Referring Image Segmentation](https://mlsanity.com/paper/2408.07539/) (2024) - [Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation](https://mlsanity.com/paper/2007.14728/) (2020) - [CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation](https://mlsanity.com/paper/2305.11481/) (2023) - [SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidance](https://mlsanity.com/paper/2409.04758/) (2024) - [Bring Adaptive Binding Prototypes to Generalized Referring Expression Segmentation](https://mlsanity.com/paper/2405.15169/) (2024) - [Language-guided Medical Image Segmentation with Target-informed Multi-level Contrastive Alignments](https://mlsanity.com/paper/2412.13533/) (2024) - [Alleviating Textual Reliance in Medical Language-guided Segmentation via Prototype-driven Semantic Approximation](https://mlsanity.com/paper/2507.11055/) (2025) ## Built on by (most cited first, 1 of 1) - [CoMLP: Cooperatively-Gated MLPs for Fine-Grained Cross-Modal Information Fusion in Medical Image Segmentation](https://mlsanity.com/paper/2609.04781/) (2026) ## Abstract (arXiv) Medical image segmentation is essential for modern computer-aided medicine. Recently, text-guided segmentation has shown promise by incorporating clinician-formulated textual reports as semantic guidance for image segmentation. These textual reports contain language descriptions about the appearance, location, and neighboring anatomy of segmentation targets, providing explicit guidance for target localization and delineation. Existing text-guided segmentation methods typically extract textual semantics implicitly through a pretrained text encoder and then integrate vision-language semantics via straightforward image-text feature fusion. However, these methods do not explicitly capture target-oriented information embedded in textual reports, particularly target location, and do not explore multi-level information fusion strategies beyond basic feature-level fusion, limiting the extraction and integration of critical textual semantics. In this study, we propose LoG, a localization-infused vision-language fusion framework for text-guided medical image segmentation. By jointly performing multi-scale target localization tasks, LoG explicitly captures target-oriented vision-language semantics and enables three-level localization-infused semantic fusion: (i) localization-guided feature fusion that directly infuses location-relevant semantics into visual features, (ii) localization-gated attention fusion that redirects multi-scale localization predictions to reinforce critical regions, and (iii) localization-constrained loss fusion that supervises segmentation based on spatial consistency with target localization. Extensive experiments on three well-established benchmark datasets, involving three medical imaging modalities with paired textual reports, demonstrate that LoG consistently outperforms state-of-the-art medical image segmentation methods.