mlsanityData to 7 Oct 2026

Medical Visual Question Answering: A Survey

arXiv 2111.10056 ↗First posted 19 Nov 2021cs.CV cs.AIcited by 93

Not on the map. Why?

Cited by 31 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Medical Visual Question Answering~(VQA) is a combination of medical artificial intelligence and popular VQA challenges. Given a medical image and a clinically relevant question in natural language, the medical VQA system is expected to predict a plausible and convincing answer. Although the general-domain VQA has been extensively studied, the medical VQA still needs specific investigation and exploration due to its task features. In the first part of this survey, we collect and discuss the publicly available medical VQA datasets up-to-date about the data source, data quantity, and task feature. In the second part, we review the approaches used in medical VQA tasks. We summarize and discuss their techniques, innovations, and potential improvements. In the last part, we analyze some medical-specific challenges for the field and discuss future research directions. Our goal is to provide comprehensive and helpful information for researchers interested in the medical visual question answering field and encourage them to conduct further research in this field.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Builds on 46 of its 116 references that are in the map. First posted on arXiv 19 Nov 2021.

Builds on

46 of 116 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  10. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  11. 2015VQA: Visual Question Answeringcited by 2,489
  12. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
34 more
  1. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  2. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  3. 2013Network In Networkcited by 600
  4. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  5. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  6. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  7. 2019BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognitioncited by 403
  8. 2015Neural Module Networkscited by 375
  9. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 371
  10. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 338
  11. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  12. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  13. 2018Bilinear Attention Networkscited by 277
  14. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  15. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  16. 2015YFCC100M: The New Data in Multimedia Researchcited by 234
  17. 2019RUBi: Reducing Unimodal Biases in Visual Question Answeringcited by 183
  18. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  19. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
  20. 2017Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysiscited by 145
  21. 2016Visual Question Answering: A Survey of Methods and Datasetscited by 123
  22. 2017Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answeringcited by 117
  23. 2016Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?cited by 111
  24. 2018Overcoming Language Priors in Visual Question Answering with Adversarial Regularizationcited by 104
  25. 2015Explicit Knowledge-based Reasoning for Visual Question Answeringcited by 85
  26. 2017Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answeringcited by 79
  27. 2020On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answeringcited by 78
  28. 2019Overcoming Data Limitation in Medical Visual Question Answeringcited by 71
  29. 2021MMBERT: Multimodal BERT Pretraining for Improved Medical VQAcited by 68
  30. 2021Multiple Meta-model Quantifying for Medical Visual Question Answeringcited by 54
  31. 2021Cross-Modal Self-Attention with Multi-Task Pre-Training for Medical Visual Question Answeringcited by 26
  32. 2020A Question-Centric Model for Visual Question Answering in Medical Imagingcited by 23
  33. 2020Hierarchical Deep Multi-modal Network for Medical Visual Question Answeringcited by 11
  34. 2019Leveraging Medical Visual Question Answering with Supporting Factscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 13; 2024: 26; 2025: 32; 2026: 20.

93 papers.

Sort:

By area:

  1. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  2. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  3. 2024Large Language Models for Robotics: Opportunities, Challenges, and Perspectivescited by 88
  4. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 84
  5. 2023Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Modelscited by 58
  6. 2023EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Imagescited by 53
  7. 2023CLIP in Medical Imaging: A Surveycited by 40
  8. 2025EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgerycited by 37
  9. 2024GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosiscited by 28
  10. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  11. 20253D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Taskscited by 24
  12. 2024Kvasir-VQA: A Text-Image Pair GI Tract Datasetcited by 23
38 more
  1. 2024PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imagingcited by 21
  2. 2024MC-CoT: A Modular Collaborative CoT Framework for Zero-shot Medical-VQA with LLM and MLLM Integrationcited by 21
  3. 2023Interpretable Medical Image Visual Question Answering via Multi-Modal Relationship Graph Learningcited by 16
  4. 2023DocLens: Multi-aspect Fine-grained Evaluation for Medical Text Generationcited by 16
  5. 2023Revisiting Distillation for Continual Learning on Visual Question Localized-Answering in Robotic Surgerycited by 15
  6. 2024Med-2E3: A 2D-Enhanced 3D Medical Multimodal Large Language Modelcited by 15
  7. 2025Are Vision Language Models Ready for Clinical Diagnosis? A 3D Medical Benchmark for Tumor-centric Visual Question Answeringcited by 14
  8. 2024LaPA: Latent Prompt Assist Model For Medical Visual Question Answeringcited by 12
  9. 2024A Survey of Medical Vision-and-Language Applications and Their Techniquescited by 11
  10. 2025mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQAcited by 10
  11. 2024Pretraining Vision-Language Model for Difference Visual Question Answering in Longitudinal Chest X-rayscited by 9
  12. 2025Open-PMC-18M: A High-Fidelity Large Scale Medical Dataset for Multimodal Representation Learningcited by 9
  13. 2023Visual Question Answering in the Medical Domaincited by 8
  14. 2025Advancing Medical Representation Learning Through High-Quality Datacited by 8
  15. 2025EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Modelscited by 8
  16. 2025Visual Question Answering on Multiple Remote Sensing Image Modalitiescited by 6
  17. 2025Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decodingcited by 6
  18. 2022A Dual-Attention Learning Network with Word and Sentence Embedding for Medical Visual Question Answeringcited by 5
  19. 2024Generalizing Visual Question Answering from Synthetic to Human-Written Questions via a Chain of QA with a Large Language Modelcited by 5
  20. 2024Fusion of Domain-Adapted Vision and Language Models for Medical Visual Question Answeringcited by 5
  21. 2024ERVQA: A Dataset to Benchmark the Readiness of Large Vision Language Models in Hospital Environmentscited by 5
  22. 2024ChitroJera: A Regionally Relevant Visual Question Answering Dataset for Banglacited by 5
  23. 2025A Study on Neuro-Symbolic Artificial Intelligence: Healthcare Perspectivescited by 5
  24. 2025MOTOR: Multimodal Optimal Transport via Grounded Retrieval in Medical Visual Question Answeringcited by 5
  25. 2024Free Form Medical Visual Question Answering in Radiologycited by 4
  26. 2024Prompt-based Personalized Federated Learning for Medical Visual Question Answeringcited by 4
  27. 2025MIRA: A Novel Framework for Fusing Modalities in Medical RAGcited by 4
  28. 2023Instance-Level Trojan Attacks on Visual Question Answering via Adversarial Learning in Neuron Activation Spacecited by 3
  29. 2024MoRE: Multi-Modal Contrastive Pre-training with Transformers on X-Rays, ECGs, and Diagnostic Reportcited by 3
  30. 2025Grounding Chest X-Ray Visual Question Answering with Generated Radiology Reportscited by 3
  31. 2025Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQAcited by 3
  32. 2025VinDr-CXR-VQA: A Visual Question Answering Dataset for Explainable Chest X-Ray Analysis with Multi-Task Learningcited by 3
  33. 2026AutoMedBench: Towards Medical AutoResearch with Agentic AI Modelscited by 3
  34. 2023Multimodal Prompt Retrieval for Generative Visual Question Answeringcited by 2
  35. 2024MediFact at MEDIQA-M3G 2024: Medical Question Answering in Dermatology with Multimodal Learningcited by 2
  36. 2025ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Imagescited by 2
  37. 2025DiN: Diffusion Model for Robust Medical VQA with Semantic Noisy Labelscited by 2
  38. 2025SurgMLLMBench: A Multimodal Large Language Model Benchmark Dataset for Surgical Scene Understandingcited by 2

50 most cited of 93 All 93 (JSON)