# Medical Visual Question Answering: A Survey

arXiv: https://arxiv.org/abs/2111.10056 · HTML: https://mlsanity.com/paper/2111.10056/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 116 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [PathVQA: 30000+ Questions for Medical Visual Question Answering](https://mlsanity.com/paper/2003.10286/) (2020)
- [SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering](https://mlsanity.com/paper/2102.09542/) (2021)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [Deep Modular Co-Attention Networks for Visual Question Answering](https://mlsanity.com/paper/1906.10770/) (2019)
- [Bilinear Attention Networks](https://mlsanity.com/paper/1805.07932/) (2018)
- [Hierarchical Question-Image Co-Attention for Visual Question Answering](https://mlsanity.com/paper/1606.00061/) (2016)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding](https://mlsanity.com/paper/1606.01847/) (2016)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [FVQA: Fact-based Visual Question Answering](https://mlsanity.com/paper/1606.05433/) (2016)

## Built on by (most cited first, 30 of 93)

- [PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering](https://mlsanity.com/paper/2305.10415/) (2023)
- [Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review](https://mlsanity.com/paper/2403.02469/) (2024)
- [Large Language Models for Robotics: Opportunities, Challenges, and Perspectives](https://mlsanity.com/paper/2401.04334/) (2024)
- [MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models](https://mlsanity.com/paper/2410.13085/) (2024)
- [Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Models](https://mlsanity.com/paper/2303.05977/) (2023)
- [EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images](https://mlsanity.com/paper/2310.18652/) (2023)
- [CLIP in Medical Imaging: A Survey](https://mlsanity.com/paper/2312.07353/) (2023)
- [EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgery](https://mlsanity.com/paper/2501.11347/) (2025)
- [GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosis](https://mlsanity.com/paper/2411.16778/) (2024)
- [3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks](https://mlsanity.com/paper/2506.11147/) (2025)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [Kvasir-VQA: A Text-Image Pair GI Tract Dataset](https://mlsanity.com/paper/2409.01437/) (2024)
- [PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging](https://mlsanity.com/paper/2401.02797/) (2024)
- [MC-CoT: A Modular Collaborative CoT Framework for Zero-shot Medical-VQA with LLM and MLLM Integration](https://mlsanity.com/paper/2410.04521/) (2024)
- [Interpretable Medical Image Visual Question Answering via Multi-Modal Relationship Graph Learning](https://mlsanity.com/paper/2302.09636/) (2023)
- [DocLens: Multi-aspect Fine-grained Evaluation for Medical Text Generation](https://mlsanity.com/paper/2311.09581/) (2023)
- [Revisiting Distillation for Continual Learning on Visual Question Localized-Answering in Robotic Surgery](https://mlsanity.com/paper/2307.12045/) (2023)
- [Med-2E3: A 2D-Enhanced 3D Medical Multimodal Large Language Model](https://mlsanity.com/paper/2411.12783/) (2024)
- [Are Vision Language Models Ready for Clinical Diagnosis? A 3D Medical Benchmark for Tumor-centric Visual Question Answering](https://mlsanity.com/paper/2505.18915/) (2025)
- [LaPA: Latent Prompt Assist Model For Medical Visual Question Answering](https://mlsanity.com/paper/2404.13039/) (2024)
- [A Survey of Medical Vision-and-Language Applications and Their Techniques](https://mlsanity.com/paper/2411.12195/) (2024)
- [mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQA](https://mlsanity.com/paper/2508.05318/) (2025)
- [Pretraining Vision-Language Model for Difference Visual Question Answering in Longitudinal Chest X-rays](https://mlsanity.com/paper/2402.08966/) (2024)
- [Open-PMC-18M: A High-Fidelity Large Scale Medical Dataset for Multimodal Representation Learning](https://mlsanity.com/paper/2506.02738/) (2025)
- [EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models](https://mlsanity.com/paper/2509.20146/) (2025)
- [Visual Question Answering in the Medical Domain](https://mlsanity.com/paper/2309.11080/) (2023)
- [Advancing Medical Representation Learning Through High-Quality Data](https://mlsanity.com/paper/2503.14377/) (2025)
- [Visual Question Answering on Multiple Remote Sensing Image Modalities](https://mlsanity.com/paper/2505.15401/) (2025)
- [Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2512.01922/) (2025)
- [A Dual-Attention Learning Network with Word and Sentence Embedding for Medical Visual Question Answering](https://mlsanity.com/paper/2210.00220/) (2022)

## Abstract (arXiv)

Medical Visual Question Answering~(VQA) is a combination of medical artificial intelligence and popular VQA challenges. Given a medical image and a clinically relevant question in natural language, the medical VQA system is expected to predict a plausible and convincing answer. Although the general-domain VQA has been extensively studied, the medical VQA still needs specific investigation and exploration due to its task features. In the first part of this survey, we collect and discuss the publicly available medical VQA datasets up-to-date about the data source, data quantity, and task feature. In the second part, we review the approaches used in medical VQA tasks. We summarize and discuss their techniques, innovations, and potential improvements. In the last part, we analyze some medical-specific challenges for the field and discuss future research directions. Our goal is to provide comprehensive and helpful information for researchers interested in the medical visual question answering field and encourage them to conduct further research in this field.