# MeDiaQA: A Question Answering Dataset on Medical Dialogues

arXiv: https://arxiv.org/abs/2108.08074 · HTML: https://mlsanity.com/paper/2108.08074/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 16 references that are in the map.
- First posted on arXiv 18 Aug 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [emrQA: A Large Corpus for Question Answering on Electronic Medical Records](https://mlsanity.com/paper/1809.00732/) (2018)
- [Interpretation of Natural Language Rules in Conversational Machine Reading](https://mlsanity.com/paper/1809.01494/) (2018)
- [Medical Exam Question Answering with Large-scale Reading Comprehension](https://mlsanity.com/paper/1802.10279/) (2018)

## Built on by (most cited first, 9 of 9)

- [PersianLLaMA: Towards Building First Persian Large Language Model](https://mlsanity.com/paper/2312.15713/) (2023)
- [Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models](https://mlsanity.com/paper/2502.18817/) (2025)
- [Conformal Information Pursuit for Interactively Guiding Large Language Models](https://mlsanity.com/paper/2507.03279/) (2025)
- [From Words and Exercises to Wellness: Farsi Chatbot for Self-Attachment Technique](https://mlsanity.com/paper/2310.09362/) (2023)
- [MedMosaic: A Challenging Large Scale Benchmark of Diverse Medical Audio](https://mlsanity.com/paper/2605.00969/) (2026)
- [HealthBranches: Synthesizing Clinically-Grounded Question Answering Datasets via Decision Pathways](https://mlsanity.com/paper/2508.07308/) (2025)
- [ThReadMed-QA: A Multi-Turn Medical Dialogue Benchmark from Real Patient Questions](https://mlsanity.com/paper/2603.11281/) (2026)
- [Question-Answering Model for Schizophrenia Symptoms and Their Impact on Daily Life using Mental Health Forums Data](https://mlsanity.com/paper/2310.00448/) (2023)
- [Can I Take Another Dose? Evaluating LLM Decision-Making Under Temporal Uncertainty in OTC Dosing QA](https://mlsanity.com/paper/2606.04262/) (2026)

## Abstract (arXiv)

In this paper, we introduce MeDiaQA, a novel question answering(QA) dataset, which constructed on real online Medical Dialogues. It contains 22k multiple-choice questions annotated by human for over 11k dialogues with 120k utterances between patients and doctors, covering 150 specialties of diseases, which are collected from haodf.com and dxy.com. MeDiaQA is the first QA dataset where reasoning over medical dialogues, especially their quantitative contents. The dataset has the potential to test the computing, reasoning and understanding ability of models across multi-turn dialogues, which is challenging compared with the existing datasets. To address the challenges, we design MeDia-BERT, and it achieves 64.3% accuracy, while human performance of 93% accuracy, which indicates that there still remains a large room for improvement.