mlsanityData to 14 Oct 2026

Beyond a Single Frame: Multi-Frame Spatially Grounded Reasoning Across Volumetric MRI

arXiv 2604.15808 ↗First posted 17 Apr 2026cs.CV cs.AIcited by 2

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

Spatial reasoning and visual grounding are core capabilities for vision-language models (VLMs), yet most medical VLMs produce predictions without transparent reasoning or spatial evidence. Existing benchmarks also evaluate VLMs on isolated 2D images, overlooking the volumetric nature of clinical imaging, where findings can span multiple frames or appear on only a few slices. We introduce Spatially Grounded MRI Visual Question Answering (SGMRI-VQA), a 41,307-pair benchmark for multi-frame, spatially grounded reasoning on volumetric MRI. Built from expert radiologist annotations in the fastMRI+ dataset across brain and knee studies, each QA pair includes a clinician-aligned chain-of-thought trace with frame-indexed bounding box coordinates. Tasks are organized hierarchically across detection, localization, counting/classification, and captioning, requiring models to jointly reason about what is present, where it is, and across which frames it extends. We benchmark 10 VLMs and show that supervised fine-tuning of Qwen3-VL-8B with bounding box supervision consistently improves grounding performance over strong zero-shot baselines, indicating that targeted spatial supervision is an effective path toward grounded clinical reasoning.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 32 of its 35 references that are in the map. First posted on arXiv 17 Apr 2026.

Builds on

32 of 35 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning22 May
  2. 2025NOVA: A Benchmark for Anomaly Localization and Clinical Reasoning in Brain MRI20 May

Other references · 30

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  8. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  9. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  10. 2021Grounded Language-Image Pre-trainingcited by 1,156
  11. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 959
  12. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
18 more
  1. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  2. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  3. 2025MedGemma Technical Reportcited by 436
  4. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  5. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  6. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
  7. 2025MedSAM2: Segment Anything in 3D Medical Images and Videoscited by 96
  8. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 64
  9. 2024Toward Visual Grounding: A Surveycited by 52
  10. 2024VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videoscited by 51
  11. 2024PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathologycited by 43
  12. 2024SegFormer3D: an Efficient Transformer for 3D Medical Image Segmentationcited by 31
  13. 2024GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosiscited by 29
  14. 2024MiniGPT-Med: Large Language Model as a General Interface for Radiology Diagnosiscited by 28
  15. 2025ReXVQA: A Large-scale Visual Question Answering Benchmark for Generalist Chest X-ray Understandingcited by 20
  16. 2025Point-It-Out: Benchmarking Embodied Reasoning for Vision Language Models in Multi-Stage Visual Groundingcited by 10
  17. 2024MedMax: Mixed-Modal Instruction Tuning for Training Biomedical Assistantscited by 6
  18. 2025VinDr-CXR-VQA: A Visual Question Answering Dataset for Explainable Chest X-Ray Analysis with Multi-Task Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Reasoning to Pixels: Grounded Medical Multimodal LLMs for VQA and Segmentation
  2. 2026Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology
  3. 2026Beyond Medical Diagnostics: How Medical Multimodal Large Language Models Think in Space
  4. 2026NeuroQA: A Large-Scale Image-Grounded Benchmark for 3D Brain MRI Understanding
  5. 2026Lost in Volume: The CT-SpatialVQA Benchmark for Evaluating Semantic-Spatial Understanding of 3D Medical Vision-Language Models
  6. 20253DReasonKnee: Advancing Grounded Reasoning in Medical Vision Language Models
  7. 2026SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering
  8. 2026Why Does Grounding Hurt Medical VQA? Benchmarking, Diagnosis, and Fine-Tuning of Vision-Language Models