# A Question-Centric Model for Visual Question Answering in Medical Imaging

arXiv: https://arxiv.org/abs/2003.08760 · HTML: https://mlsanity.com/paper/2003.08760/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2020.

## Built on by (most cited first, 23 of 23)

- [Medical Visual Question Answering: A Survey](https://mlsanity.com/paper/2111.10056/) (2021)
- [Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?](https://mlsanity.com/paper/2112.13906/) (2021)
- [Multiple Meta-model Quantifying for Medical Visual Question Answering](https://mlsanity.com/paper/2105.08913/) (2021)
- [Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery](https://mlsanity.com/paper/2408.04958/) (2024)
- [VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges](https://mlsanity.com/paper/2212.13296/) (2022)
- [A Survey of Natural Language Generation](https://mlsanity.com/paper/2112.11739/) (2021)
- [Visual Question Answering: which investigated applications?](https://mlsanity.com/paper/2103.02937/) (2021)
- [Consistency-preserving Visual Question Answering in Medical Imaging](https://mlsanity.com/paper/2206.13296/) (2022)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [UnICLAM:Contrastive Representation Learning with Adversarial Masking for Unified and Interpretable Medical Vision Question Answering](https://mlsanity.com/paper/2212.10729/) (2022)
- [Logical Implications for Visual Question Answering Consistency](https://mlsanity.com/paper/2303.09427/) (2023)
- [A Dual-Attention Learning Network with Word and Sentence Embedding for Medical Visual Question Answering](https://mlsanity.com/paper/2210.00220/) (2022)
- [Localized Questions in Medical Visual Question Answering](https://mlsanity.com/paper/2307.01067/) (2023)
- [MuVAM: A Multi-View Attention-based Model for Medical Visual Question Answering](https://mlsanity.com/paper/2107.03216/) (2021)
- [Free Form Medical Visual Question Answering in Radiology](https://mlsanity.com/paper/2401.13081/) (2024)
- [LOIS: Looking Out of Instance Semantics for Visual Question Answering](https://mlsanity.com/paper/2307.14142/) (2023)
- [PiggyBack: Pretrained Visual Question Answering Environment for Backing up Non-deep Learning Professionals](https://mlsanity.com/paper/2211.15940/) (2022)
- [A Unified Framework for Tabular Generative Modeling: Loss Functions, Benchmarks, and Improved Multi-objective Bayesian Optimization Approaches](https://mlsanity.com/paper/2405.16971/) (2024)
- [Tri-VQA: Triangular Reasoning Medical Visual Question Answering for Multi-Attribute Analysis](https://mlsanity.com/paper/2406.15050/) (2024)
- [Targeted Visual Prompting for Medical Visual Question Answering](https://mlsanity.com/paper/2408.03043/) (2024)
- [Cause-Effect Driven Optimization for Robust Medical Visual Question Answering with Language Biases](https://mlsanity.com/paper/2506.17903/) (2025)
- [Barriers in Integrating Medical Visual Question Answering into Radiology Workflows: A Scoping Review and Clinicians' Insights](https://mlsanity.com/paper/2507.08036/) (2025)
- [A reinforcement learning approach for VQA validation: an application to diabetic macular edema grading](https://mlsanity.com/paper/2307.09886/) (2023)

## Abstract (arXiv)

Deep learning methods have proven extremely effective at performing a variety of medical image analysis tasks. With their potential use in clinical routine, their lack of transparency has however been one of their few weak points, raising concerns regarding their behavior and failure modes. While most research to infer model behavior has focused on indirect strategies that estimate prediction uncertainties and visualize model support in the input image space, the ability to explicitly query a prediction model regarding its image content offers a more direct way to determine the behavior of trained models. To this end, we present a novel Visual Question Answering approach that allows an image to be queried by means of a written question. Experiments on a variety of medical and natural image datasets show that by fusing image and question features in a novel way, the proposed approach achieves an equal or higher accuracy compared to current methods.