mlsanityData to 7 Oct 2026

PathVQA: 30000+ Questions for Medical Visual Question Answering

arXiv 2003.10286 ↗First posted 7 Mar 2020cs.CL cs.AIcited by 467

Not on the map. Why?

Cited by 225 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Is it possible to develop an "AI Pathologist" to pass the board-certified examination of the American Board of Pathology? To achieve this goal, the first step is to create a visual question answering (VQA) dataset where the AI agent is presented with a pathology image together with a question and is asked to give the correct answer. Our work makes the first attempt to build such a dataset. Different from creating general-domain VQA datasets where the images are widely accessible and there are many crowdsourcing workers available and capable of generating question-answer pairs, developing a medical VQA dataset is much more challenging. First, due to privacy concerns, pathology images are usually not publicly available. Second, only well-trained pathologists can understand pathology images, but they barely have time to help create datasets for AI research. To address these challenges, we resort to pathology textbooks and online digital libraries. We develop a semi-automated pipeline to extract pathology images and captions from textbooks and generate question-answer pairs from captions using natural language processing. We collect 32,799 open-ended questions from 4,998 pathology images where each question is manually checked to ensure correctness. To our best knowledge, this is the first dataset for pathology VQA. Our dataset will be released publicly to promote research in medical VQA.

Key facts. Cited by 467 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 27; 2024: 89; 2025: 166; 2026: 171.

467 papers.

Sort:

By area:

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  2. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  3. 2023A Survey on Multimodal Large Language Modelscited by 898
  4. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  5. 2024What matters when building vision-language models?cited by 321
  6. 2023Towards Generalist Biomedical AIcited by 286
  7. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 217
  8. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 212
  9. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  10. 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 194
  11. 2023Quilt-1M: One Million Image-Text Pairs for Histopathologycited by 192
  12. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 182
38 more
  1. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learningcited by 172
  2. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  3. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  4. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  5. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  6. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  7. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  8. 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolutioncited by 114
  9. 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 112
  10. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agentcited by 108
  11. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  12. 2023Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuningcited by 106
  13. 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecturecited by 100
  14. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
  15. 2021Medical Visual Question Answering: A Surveycited by 93
  16. 2024Advancing Multimodal Medical Capabilities of Geminicited by 88
  17. 2023Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videoscited by 84
  18. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicinecited by 79
  19. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  20. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  21. 2025MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflowcited by 71
  22. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 69
  23. 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoningcited by 69
  24. 2021MMBERT: Multimodal BERT Pretraining for Improved Medical VQAcited by 68
  25. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  26. 2023Open-Ended Medical Visual Question Answering Through Prefix Tuning of Language Modelscited by 58
  27. 2023PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathologycited by 57
  28. 2021Multiple Meta-model Quantifying for Medical Visual Question Answeringcited by 54
  29. 2025FineVision: Open Data Is All You Needcited by 51
  30. 2024VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledgecited by 50
  31. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  32. 2024MedCoT: Medical Chain of Thought via Hierarchical Expertcited by 46
  33. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  34. 2024PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathologycited by 43
  35. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Modelscited by 43
  36. 2025Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Trainingcited by 42
  37. 2025MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaborationcited by 39
  38. 2023Multimodal ChatGPT for Medical Applications: an Experimental Study of GPT-4Vcited by 38

50 most cited of 467 All 467 (JSON)