# CliCR: A Dataset of Clinical Case Reports for Machine Reading Comprehension

arXiv: https://arxiv.org/abs/1803.09720 · HTML: https://mlsanity.com/paper/1803.09720/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2018.

## Built on by (most cited first, 30 of 34)

- [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://mlsanity.com/paper/2203.14371/) (2022)
- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [A Survey of Large Language Models in Medicine: Progress, Application, and Challenge](https://mlsanity.com/paper/2311.05112/) (2023)
- [CMB: A Comprehensive Medical Benchmark in Chinese](https://mlsanity.com/paper/2308.08833/) (2023)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset](https://mlsanity.com/paper/2306.03030/) (2023)
- [Pre-trained Language Models in Biomedical Domain: A Systematic Survey](https://mlsanity.com/paper/2110.05006/) (2021)
- [Towards Mitigating Hallucination in Large Language Models via Self-Reflection](https://mlsanity.com/paper/2310.06271/) (2023)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks](https://mlsanity.com/paper/2407.12994/) (2024)
- [Biomedical Question Answering: A Survey of Approaches and Challenges](https://mlsanity.com/paper/2102.05281/) (2021)
- [Huatuo-26M, a Large-scale Chinese Medical QA Dataset](https://mlsanity.com/paper/2305.01526/) (2023)
- [AMMU : A Survey of Transformer-based Biomedical Pretrained Language Models](https://mlsanity.com/paper/2105.00827/) (2021)
- [Conversational Question Answering: A Survey](https://mlsanity.com/paper/2106.00874/) (2021)
- [English Machine Reading Comprehension Datasets: A Survey](https://mlsanity.com/paper/2101.10421/) (2021)
- [Modern Question Answering Datasets and Benchmarks: A Survey](https://mlsanity.com/paper/2206.15030/) (2022)
- [DrugEHRQA: A Question Answering Dataset on Structured and Unstructured Electronic Health Records For Medicine Related Queries](https://mlsanity.com/paper/2205.01290/) (2022)
- [Large Language Model Benchmarks in Medical Tasks](https://mlsanity.com/paper/2410.21348/) (2024)
- [Learning to Ask Like a Physician](https://mlsanity.com/paper/2206.02696/) (2022)
- [Question Answering for Complex Electronic Health Records Database using Unified Encoder-Decoder Architecture](https://mlsanity.com/paper/2111.14703/) (2021)
- [SyllabusQA: A Course Logistics Question Answering Dataset](https://mlsanity.com/paper/2403.14666/) (2024)
- [Question Answering for Electronic Health Records: A Scoping Review of datasets and models](https://mlsanity.com/paper/2310.08759/) (2023)
- [SPBERTQA: A Two-Stage Question Answering System Based on Sentence Transformers for Medical Texts](https://mlsanity.com/paper/2206.09600/) (2022)
- [To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering](https://mlsanity.com/paper/2212.10381/) (2022)
- [Attention-based Aspect Reasoning for Knowledge Base Question Answering on Clinical Notes](https://mlsanity.com/paper/2108.00513/) (2021)
- [Uncertainty-Aware Text-to-Program for Question Answering on Structured Electronic Health Records](https://mlsanity.com/paper/2203.06918/) (2022)
- [SA-MDKIF: A Scalable and Adaptable Medical Domain Knowledge Injection Framework for Large Language Models](https://mlsanity.com/paper/2402.00474/) (2024)
- [Can GPT Redefine Medical Understanding? Evaluating GPT on Biomedical Machine Reading Comprehension](https://mlsanity.com/paper/2405.18682/) (2024)
- [Multi Document Reading Comprehension](https://mlsanity.com/paper/2201.01706/) (2022)
- [InDEX: Indonesian Idiom and Expression Dataset for Cloze Test](https://mlsanity.com/paper/2211.13376/) (2022)

## Abstract (arXiv)

We present a new dataset for machine comprehension in the medical domain. Our dataset uses clinical case reports with around 100,000 gap-filling queries about these cases. We apply several baselines and state-of-the-art neural readers to the dataset, and observe a considerable gap in performance (20% F1) between the best human and machine readers. We analyze the skills required for successful answering and show how reader performance varies depending on the applicable skills. We find that inferences using domain knowledge and object tracking are the most frequently required skills, and that recognizing omitted information and spatio-temporal reasoning are the most difficult for the machines.