# Localized Questions in Medical Visual Question Answering

arXiv: https://arxiv.org/abs/2307.01067 · HTML: https://mlsanity.com/paper/2307.01067/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering](https://mlsanity.com/paper/2102.09542/) (2021)
- [2017 Robotic Instrument Segmentation Challenge](https://mlsanity.com/paper/1902.06426/) (2019)
- [MUTAN: Multimodal Tucker Fusion for Visual Question Answering](https://mlsanity.com/paper/1705.06676/) (2017)
- [GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering](https://mlsanity.com/paper/1902.09506/) (2019)
- [Overcoming Data Limitation in Medical Visual Question Answering](https://mlsanity.com/paper/1909.11867/) (2019)
- [Multiple Meta-model Quantifying for Medical Visual Question Answering](https://mlsanity.com/paper/2105.08913/) (2021)
- [Point and Ask: Incorporating Pointing into Visual Question Answering](https://mlsanity.com/paper/2011.13681/) (2020)
- [Cross-Modal Self-Attention with Multi-Task Pre-Training for Medical Visual Question Answering](https://mlsanity.com/paper/2105.00136/) (2021)
- [A Question-Centric Model for Visual Question Answering in Medical Imaging](https://mlsanity.com/paper/2003.08760/) (2020)
- [Consistency-preserving Visual Question Answering in Medical Imaging](https://mlsanity.com/paper/2206.13296/) (2022)

## Built on by (most cited first, 5 of 5)

- [Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery](https://mlsanity.com/paper/2408.04958/) (2024)
- [Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMs](https://mlsanity.com/paper/2505.00744/) (2025)
- [A Survey of Medical Vision-and-Language Applications and Their Techniques](https://mlsanity.com/paper/2411.12195/) (2024)
- [Targeted Visual Prompting for Medical Visual Question Answering](https://mlsanity.com/paper/2408.03043/) (2024)
- [Dual Causal Inference: Integrating Backdoor Adjustment and Instrumental Variable Learning for Medical VQA](https://mlsanity.com/paper/2604.20306/) (2026)

## Abstract (arXiv)

Visual Question Answering (VQA) models aim to answer natural language questions about given images. Due to its ability to ask questions that differ from those used when training the model, medical VQA has received substantial attention in recent years. However, existing medical VQA models typically focus on answering questions that refer to an entire image rather than where the relevant content may be located in the image. Consequently, VQA models are limited in their interpretability power and the possibility to probe the model about specific image regions. This paper proposes a novel approach for medical VQA that addresses this limitation by developing a model that can answer questions about image regions while considering the context necessary to answer the questions. Our experimental results demonstrate the effectiveness of our proposed model, outperforming existing methods on three datasets. Our code and data are available at https://github.com/sergiotasconmorales/locvqa.