mlsanityData to 7 Oct 2026

What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams

arXiv 2009.13081 ↗First posted 28 Sep 2020cs.CL cs.AIcited by 1,324

Not on the map. Why?

Cited by 201 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Open domain question answering (OpenQA) tasks have been recently attracting more and more attention from the natural language processing (NLP) community. In this work, we present the first free-form multiple-choice OpenQA dataset for solving medical problems, MedQA, collected from the professional medical board exams. It covers three languages: English, simplified Chinese, and traditional Chinese, and contains 12,723, 34,251, and 14,123 questions for the three languages, respectively. We implement both rule-based and popular neural methods by sequentially combining a document retriever and a machine comprehension model. Through experiments, we find that even the current best method can only achieve 36.7%, 42.0%, and 70.1% of test accuracy on the English, traditional Chinese, and simplified Chinese questions, respectively. We expect MedQA to present great challenges to existing OpenQA systems and hope that it can serve as a platform to promote much stronger OpenQA models from the NLP community in the future.

Key facts. Cited by 1,324 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 13; 2023: 89; 2024: 276; 2025: 436; 2026: 504.

1,324 papers.

Sort:

By area:

  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  3. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  4. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  5. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 386
  6. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
  7. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  8. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  9. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  10. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  11. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  12. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
38 more
  1. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  2. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 299
  3. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 295
  4. 2023Towards Generalist Biomedical AIcited by 286
  5. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  6. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 272
  7. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
  8. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  9. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 217
  10. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environmentscited by 215
  11. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  12. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 212
  13. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 210
  14. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  15. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
  16. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  17. 2022Can large language models reason about medical questions?cited by 194
  18. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 189
  19. 2024Large Language Models for Education: A Survey and Outlookcited by 185
  20. 2024Are We Done with MMLU?cited by 178
  21. 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 170
  22. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  23. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 162
  24. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 160
  25. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  26. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  27. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  28. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 145
  29. 2023PaperQA: Retrieval-Augmented Generative Agent for Scientific Researchcited by 143
  30. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 124
  31. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  32. 2023Almanac: Retrieval-Augmented Language Models for Clinical Medicinecited by 122
  33. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  34. 2023Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Healthcited by 118
  35. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  36. 2023Towards Accurate Differential Diagnosis with Large Language Modelscited by 115
  37. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 113
  38. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 112

50 most cited of 1,324 All 1,324 (JSON)