mlsanityData to 7 Oct 2026

Medical Image Spatial Grounding with Semantic Sampling

arXiv 2603.14579 ↗First posted 15 Mar 2026cs.CV cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision language models (VLMs) have shown significant promise in visual grounding for images as well as videos. In medical imaging research, VLMs represent a bridge between object detection and segmentation, and report understanding and generation. However, spatial grounding of anatomical structures in the three-dimensional space of medical images poses many unique challenges. In this study, we examine image modalities, slice directions, and coordinate systems as differentiating factors for vision components of VLMs, and the use of anatomical, directional, and relational terminology as factors for the language components. We then demonstrate that visual and textual prompting systems such as labels, bounding boxes, and mask overlays have varying effects on the spatial grounding ability of VLMs. To enable measurement and reproducibility, we introduce MIS-Ground, a benchmark that comprehensively tests a VLM for vulnerabilities against specific modes of Medical Image Spatial Grounding. We release MIS-Ground to the public at github.com/asy51/mis-ground. In addition, we present MIS-SemSam, a low-cost, inference-time, and model-agnostic optimization of VLMs that improves their spatial grounding ability with the use of Semantic Sampling. We find that MIS-SemSam improves the accuracy of Qwen3-VL-32B on MIS-Ground by 13.06%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 25 references that are in the map. First posted on arXiv 15 Mar 2026.

Builds on

20 of 25 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026AnatomiX, an Anatomy-Aware Grounded Multimodal Large Language Model for Chest X-Ray Interpretation6 Jan
  2. 20253DReasonKnee: Advancing Grounded Reasoning in Medical Vision Language Models23 Oct
  3. 2025Your other Left! Vision-Language Models Fail to Identify Relative Positions in Medical Images1 Aug
  4. 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis25 Mar
  5. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models31 Mar

Other references · 15

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  7. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  8. 2022TotalSegmentator: robust segmentation of 104 anatomical structures in CT imagescited by 597
  9. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  10. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  11. 2025MedGemma Technical Reportcited by 398
  12. 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 192
3 more
  1. 2026Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Groundingcited by 132
  2. 2025Don't Pass@k: A Bayesian Framework for Large Language Model Evaluationcited by 15
  3. 2024Novel adaptation of video segmentation to 3D MRI: efficient zero-shot knee segmentation with SAM2cited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images
  2. 2026LocAnyMed: Vision-Language Grounding for Multimodal Medical Images
  3. 2026Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology
  4. 2025MedSG-Bench: A Benchmark for Medical Image Sequences Grounding
  5. 2026MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data
  6. 2024VividMed: Vision Language Model with Versatile Visual Grounding for Medicine
  7. 2025Enhancing Abnormality Grounding for Vision Language Models with Knowledge Descriptions
  8. 2026Beyond a Single Frame: Multi-Frame Spatially Grounded Reasoning Across Volumetric MRI