# Annotating Electronic Medical Records for Question Answering

arXiv: https://arxiv.org/abs/1805.06816 · HTML: https://mlsanity.com/paper/1805.06816/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2018.

## Built on by (most cited first, 6 of 6)

- [MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records](https://mlsanity.com/paper/2308.14089/) (2023)
- [Biomedical Question Answering: A Survey of Approaches and Challenges](https://mlsanity.com/paper/2102.05281/) (2021)
- [Evaluation of AI Chatbots for Patient-Specific EHR Questions](https://mlsanity.com/paper/2306.02549/) (2023)
- [Question Answering for Electronic Health Records: A Scoping Review of datasets and models](https://mlsanity.com/paper/2310.08759/) (2023)
- [Development of an Extractive Clinical Question Answering Dataset with Multi-Answer and Multi-Focus Questions](https://mlsanity.com/paper/2201.02517/) (2022)
- [Towards More Robust Natural Language Understanding](https://mlsanity.com/paper/2112.02992/) (2021)

## Abstract (arXiv)

Our research is in the relatively unexplored area of question answering technologies for patient-specific questions over their electronic health records. A large dataset of human expert curated question and answer pairs is an important pre-requisite for developing, training and evaluating any question answering system that is powered by machine learning. In this paper, we describe a process for creating such a dataset of questions and answers. Our methodology is replicable, can be conducted by medical students as annotators, and results in high inter-annotator agreement (0.71 Cohen's kappa). Over the course of 11 months, 11 medical students followed our annotation methodology, resulting in a question answering dataset of 5696 questions over 71 patient records, of which 1747 questions have corresponding answers generated by the medical students.