mlsanityData to 14 Oct 2026

Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation

arXiv 2607.16327 ↗First posted 15 Jul 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Medical image segmentation is essential for modern computer-aided medicine. Recently, text-guided segmentation has shown promise by incorporating clinician-formulated textual reports as semantic guidance for image segmentation. These textual reports contain language descriptions about the appearance, location, and neighboring anatomy of segmentation targets, providing explicit guidance for target localization and delineation. Existing text-guided segmentation methods typically extract textual semantics implicitly through a pretrained text encoder and then integrate vision-language semantics via straightforward image-text feature fusion. However, these methods do not explicitly capture target-oriented information embedded in textual reports, particularly target location, and do not explore multi-level information fusion strategies beyond basic feature-level fusion, limiting the extraction and integration of critical textual semantics. In this study, we propose LoG, a localization-infused vision-language fusion framework for text-guided medical image segmentation. By jointly performing multi-scale target localization tasks, LoG explicitly captures target-oriented vision-language semantics and enables three-level localization-infused semantic fusion: (i) localization-guided feature fusion that directly infuses location-relevant semantics into visual features, (ii) localization-gated attention fusion that redirects multi-scale localization predictions to reinforce critical regions, and (iii) localization-constrained loss fusion that supervises segmentation based on spatial consistency with target localization. Extensive experiments on three well-established benchmark datasets, involving three medical imaging modalities with paired textual reports, demonstrate that LoG consistently outperforms state-of-the-art medical image segmentation methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 55 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

32 of 55 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2022A ConvNet for the 2020scited by 3,579
  5. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  6. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,525
  7. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,435
  8. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
  9. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  10. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  11. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 589
  12. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
20 more
  1. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
  2. 2022MAXIM: Multi-Axis MLP for Image Processingcited by 302
  3. 2022LViT: Language meets Vision Transformer in Medical Image Segmentationcited by 111
  4. 2020MosMedData: Chest CT Scans With COVID-19 Related Findings Datasetcited by 104
  5. 2023ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detectioncited by 76
  6. 2022OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Imagescited by 48
  7. 2021A Survey on Ensemble Learning under the Era of Deep Learningcited by 47
  8. 2023Ariadne's Thread:Using Text Prompts to Improve Segmentation of Infected Areas from Chest X-ray imagescited by 34
  9. 2023Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Modelscited by 28
  10. 2024Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentationcited by 28
  11. 2024Task-driven Image Fusion with Learnable Fusion Losscited by 25
  12. 2024Cross-aware Early Fusion with Stage-divided Vision and Language Transformer Encoders for Referring Image Segmentationcited by 21
  13. 2020Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentationcited by 19
  14. 2023CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentationcited by 15
  15. 2024SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidancecited by 11
  16. 2024Bring Adaptive Binding Prototypes to Generalized Referring Expression Segmentationcited by 8
  17. 2024Language-guided Medical Image Segmentation with Target-informed Multi-level Contrastive Alignmentscited by 6
  18. 2025Alleviating Textual Reliance in Medical Language-guided Segmentation via Prototype-driven Semantic Approximationcited by 6
  19. 2025Frequency-domain Multi-modal Fusion for Language-guided Medical Image Segmentationcited by 4
  20. 2025MCFNet: A Multimodal Collaborative Fusion Network for Fine-Grained Semantic Classificationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CoMLP: Cooperatively-Gated MLPs for Fine-Grained Cross-Modal Information Fusion in Medical Image Segmentationcited by 0