mlsanityData to 7 Oct 2026

BIOMRC: A Dataset for Biomedical Machine Reading Comprehension

arXiv 2005.06376 ↗First posted 13 May 2020cs.CL cs.LG stat.MLcited by 35

Not on the map. Why?

Abstract (arXiv)

We introduce BIOMRC, a large-scale cloze-style biomedical MRC dataset. Care was taken to reduce noise, compared to the previous BIOREAD dataset of Pappas et al. (2018). Experiments show that simple heuristics do not perform well on the new dataset, and that two neural MRC models that had been tested on BIOREAD perform much better on BIOMRC, indicating that the new dataset is indeed less noisy or at least that its task is more feasible. Non-expert human performance is also higher on the new dataset compared to BIOREAD, and biomedical experts perform even better. We also introduce a new BERT-based MRC model, the best version of which substantially outperforms all other methods tested, reaching or surpassing the accuracy of biomedical experts in some experiments. We make the new dataset available in three different sizes, also releasing our code, and providing a leaderboard.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 7; 2023: 6; 2024: 11; 2025: 3; 2026: 1.

35 papers.

Sort:

By area:

  1. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  2. 2021MetaICL: Learning to Learn In Contextcited by 450
  3. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  4. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  5. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  6. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  7. 2024A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Taskscited by 46
  8. 2022BigBIO: A Framework for Data-Centric Biomedical Natural Language Processingcited by 36
  9. 2021Exploring Universal Intrinsic Task Subspace via Prompt Tuningcited by 25
  10. 2024SciDQA: A Deep Reading Comprehension Dataset over Scientific Paperscited by 15
  11. 2023Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Taskscited by 13
  12. 2021Sequence tagging for biomedical extractive question answeringcited by 12
23 more
  1. 2022Eliciting and Understanding Cross-Task Skills with Task-Level Mixture-of-Expertscited by 12
  2. 2023Learning to Initialize: Can Meta Learning Improve Cross-task Generalization in Prompt Tuning?cited by 11
  3. 2024SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Typescited by 10
  4. 2023Can LLMs Augment Low-Resource Reading Comprehension Datasets? Opportunities and Challengescited by 9
  5. 2024MoDEM: Mixture of Domain Expert Modelscited by 9
  6. 2023NuclearQA: A Human-Made Benchmark for Language Models for the Nuclear Domaincited by 7
  7. 2022Few-shot Adaptation Works with UnpredicTable Datacited by 6
  8. 2022Data Augmentation for Biomedical Factoid Question Answeringcited by 5
  9. 2024Comparing Template-based and Template-free Language Model Probingcited by 5
  10. 2022Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answeringcited by 4
  11. 2024WeQA: A Benchmark for Retrieval Augmented Generation in Wind Energy Domaincited by 4
  12. 2024M-QALM: A Benchmark to Assess Clinical Reading Comprehension and Knowledge Recall in Large Language Models via Question Answeringcited by 3
  13. 2024Optimized Biomedical Question-Answering Services with LLM and Multi-BERT Integrationcited by 2
  14. 2025From Queries to Criteria: Understanding How Astronomers Evaluate LLMscited by 2
  15. 2022ArgSciChat: A Dataset for Argumentative Dialogues on Scientific Paperscited by 1
  16. 2022KnowDA: All-in-One Knowledge Mixture Model for Data Augmentation in Low-Resource NLPcited by 1
  17. 2024Can GPT Redefine Medical Understanding? Evaluating GPT on Biomedical Machine Reading Comprehensioncited by 1
  18. 2021Recent Advances in Automated Question Answering In Biomedical Domaincited by 0
  19. 2023SciMRC: Multi-perspective Scientific Machine Reading Comprehensioncited by 0
  20. 2024Recent Advances in Multi-Choice Machine Reading Comprehension: A Survey on Methods and Datasetscited by 0
  21. 2025IMB: An Italian Medical Benchmark for Question Answeringcited by 0
  22. 2025Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answeringcited by 0
  23. 2026Drift-Aware Temporal Graph Rewiring (DATGR) for Adaptive Semantic Modeling in Biomedical Textcited by 0