# Learning Where to Look: Anatomical Grounding and Guided Attention for Cardiac MRI Vision-Language Models

arXiv: https://arxiv.org/abs/2609.39899 · HTML: https://mlsanity.com/paper/2609.39899/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 16 of 16)

- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Med-Flamingo: a Multimodal Medical Few-shot Learner](https://mlsanity.com/paper/2307.15189/) (2023)
- [Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning](https://mlsanity.com/paper/2506.07044/) (2025)
- [Multivariate mixture model for myocardium segmentation combining multi-source images](https://mlsanity.com/paper/1612.08820/) (2016)
- [UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation](https://mlsanity.com/paper/2107.00781/) (2021)
- [MedGemma 1.5 Technical Report](https://mlsanity.com/paper/2604.05081/) (2026)
- [MyoPS-Net: Myocardial Pathology Segmentation with Flexible Combination of Multi-Sequence CMR Images](https://mlsanity.com/paper/2211.03062/) (2022)
- [A Generalizable Deep Learning System for Cardiac MRI](https://mlsanity.com/paper/2312.00357/) (2023)
- [Aligning Multi-Sequence CMR Towards Fully Automated Myocardial Pathology Segmentation](https://mlsanity.com/paper/2302.03537/) (2023)
- [MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data](https://mlsanity.com/paper/2601.06847/) (2026)
- [VerSe: Integrating Multiple Queries as Prompts for Versatile Cardiac MRI Segmentation](https://mlsanity.com/paper/2412.16381/) (2024)
- [CineMyoPS: Segmenting Myocardial Pathologies from Cine Cardiac MR](https://mlsanity.com/paper/2507.02289/) (2025)
- [K-Prism: A Knowledge-Guided and Prompt Integrated Universal Medical Image Segmentation Model](https://mlsanity.com/paper/2509.25594/) (2025)
- [Anatomy-VLM: A Fine-grained Vision-Language Model for Medical Interpretation](https://mlsanity.com/paper/2511.08402/) (2025)
- [BAAI Cardiac Agent: An intelligent multimodal agent for automated reasoning and diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging](https://mlsanity.com/paper/2604.04078/) (2026)
- [Learning Volumetric Neural Deformable Models to Recover 3D Regional Heart Wall Motion from Multi-Planar Tagged MRI](https://mlsanity.com/paper/2411.15233/) (2024)

## Abstract (arXiv)

Cardiac magnetic resonance imaging (CMR) enables assessment of cardiac anatomy, ventricular function, and myocardial tissue characteristics. Clinicians interpret these images by identifying cardiac structures and focusing on the regions relevant to each clinical question, motivating anatomically guided vision-language models (VLMs). Yet CMR-specific supervision for anatomical localisation and clinical question answering remains limited. To address this gap, we investigate fine-grained CMR visual question answering through anatomical grounding and guided attention. We construct 128,915 anatomical-grounding and 42,799 clinical QA pairs across short-axis cine, late gadolinium enhancement, and long-axis cine. These datasets support anatomical recognition, localisation, and clinical assessment without requiring paired reports for individual training images. To help the model learn where to look, we introduce Cardiac Anatomy-Routed Attention (CARA), which selects predicted anatomical priors according to the question and guides decoder attention with learned task-specific strengths. Combining anatomical grounding pretraining with CARA yields our model, CARA-VL. Experiments demonstrate CARA-VL's strengths in clinical assessment and regional localisation across CMR imaging settings, with promising generalization to an external clinical cohort. Together, our data and method provide a practical framework for studying and advancing cardiac visual understanding in VLMs. We will release the QA data derived from public datasets upon publication.