# What Would it Take to get Biomedical QA Systems into Practice?

arXiv: https://arxiv.org/abs/2109.10415 · HTML: https://mlsanity.com/paper/2109.10415/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 45 references that are in the map.
- First posted on arXiv 21 Sep 2021.

## Builds on (most cited first, 9 of 9)

- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [emrQA: A Large Corpus for Question Answering on Electronic Medical Records](https://mlsanity.com/paper/1809.00732/) (2018)
- [Inferring Which Medical Treatments Work from Reports of Clinical Trials](https://mlsanity.com/paper/1904.01606/) (2019)
- [Generating (Factual?) Narrative Summaries of RCTs: Experiments with Neural Multi-Document Summarization](https://mlsanity.com/paper/2008.11293/) (2020)
- [Question-Driven Summarization of Answers to Consumer Health Questions](https://mlsanity.com/paper/2005.09067/) (2020)
- [Results of the seventh edition of the BioASQ Challenge](https://mlsanity.com/paper/2006.09174/) (2020)
- [Understanding Clinical Trial Reports: Extracting Medical Entities and Their Relations](https://mlsanity.com/paper/2010.03550/) (2020)
- [CLINIQA: A Machine Intelligence Based Clinical Question Answering System](https://mlsanity.com/paper/1805.05927/) (2018)
- [Nutribullets Hybrid: Multi-document Health Summarization](https://mlsanity.com/paper/2104.03465/) (2021)

## Built on by (most cited first, 7 of 7)

- [Learning to Ask Like a Physician](https://mlsanity.com/paper/2206.02696/) (2022)
- [Do Multi-Document Summarization Models Synthesize?](https://mlsanity.com/paper/2301.13844/) (2023)
- [RealMedQA: A pilot biomedical question answering dataset containing realistic clinical questions](https://mlsanity.com/paper/2408.08624/) (2024)
- [Question answering systems for health professionals at the point of care -- a systematic review](https://mlsanity.com/paper/2402.01700/) (2024)
- [Using contradictions improves question answering systems](https://mlsanity.com/paper/2211.05598/) (2022)
- [Emerging Challenges in Personalized Medicine: Assessing Demographic Effects on Biomedical Question Answering Systems](https://mlsanity.com/paper/2310.10571/) (2023)
- [Knowledge Models for Cancer Clinical Practice Guidelines : Construction, Management and Usage in Question Answering](https://mlsanity.com/paper/2407.21053/) (2024)

## Abstract (arXiv)

Medical question answering (QA) systems have the potential to answer clinicians uncertainties about treatment and diagnosis on demand, informed by the latest evidence. However, despite the significant progress in general QA made by the NLP community, medical QA systems are still not widely used in clinical environments. One likely reason for this is that clinicians may not readily trust QA system outputs, in part because transparency, trustworthiness, and provenance have not been key considerations in the design of such models. In this paper we discuss a set of criteria that, if met, we argue would likely increase the utility of biomedical QA systems, which may in turn lead to adoption of such systems in practice. We assess existing models, tasks, and datasets with respect to these criteria, highlighting shortcomings of previously proposed approaches and pointing toward what might be more usable QA systems.