mlsanityData to 7 Oct 2026

Question answering systems for health professionals at the point of care -- a systematic review

arXiv 2402.01700 ↗First posted 24 Jan 2024cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Objective: Question answering (QA) systems have the potential to improve the quality of clinical care by providing health professionals with the latest and most relevant evidence. However, QA systems have not been widely adopted. This systematic review aims to characterize current medical QA systems, assess their suitability for healthcare, and identify areas of improvement. Materials and methods: We searched PubMed, IEEE Xplore, ACM Digital Library, ACL Anthology and forward and backward citations on 7th February 2023. We included peer-reviewed journal and conference papers describing the design and evaluation of biomedical QA systems. Two reviewers screened titles, abstracts, and full-text articles. We conducted a narrative synthesis and risk of bias assessment for each study. We assessed the utility of biomedical QA systems. Results: We included 79 studies and identified themes, including question realism, answer reliability, answer utility, clinical specialism, systems, usability, and evaluation methods. Clinicians' questions used to train and evaluate QA systems were restricted to certain sources, types and complexity levels. No system communicated confidence levels in the answers or sources. Many studies suffered from high risks of bias and applicability concerns. Only 8 studies completely satisfied any criterion for clinical utility, and only 7 reported user evaluations. Most systems were built with limited input from clinicians. Discussion: While machine learning methods have led to increased accuracy, most studies imperfectly reflected real-world healthcare information needs. Key research priorities include developing more realistic healthcare QA datasets and considering the reliability of answer sources, rather than merely focusing on accuracy.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 32 of its 123 references that are in the map. First posted on arXiv 24 Jan 2024.

Builds on

32 of 123 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  8. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2021Ethical and social risks of harm from Language Modelscited by 957
  11. 2022Galactica: A Large Language Model for Sciencecited by 736
  12. 2019A Question-Entailment Approach to Question Answeringcited by 133
20 more
  1. 2020BioMegatron: Larger Biomedical Domain Language Modelcited by 76
  2. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  3. 2023Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)cited by 40
  4. 2023Diving Deep into Modes of Fact Hallucinations in Dialogue Systemscited by 32
  5. 2017Neural Domain Adaptation for Biomedical Question Answeringcited by 30
  6. 2021Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategiescited by 21
  7. 2019Unsupervised Domain Adaptation of Language Models for Reading Comprehensioncited by 15
  8. 2021Benchmarking for Biomedical Natural Language Processing Tasks with a Domain Specific ALBERTcited by 14
  9. 2021Sequence tagging for biomedical extractive question answeringcited by 12
  10. 2020F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answeringcited by 11
  11. 2021What Would it Take to get Biomedical QA Systems into Practice?cited by 7
  12. 2016Using Centroids of Word Embeddings and Word Mover's Distance for Biomedical Document Retrieval in Question Answeringcited by 6
  13. 2019Adapting and evaluating a deep learning language model for clinical why-question answeringcited by 5
  14. 2021Task-adaptive Pre-training of Language Models with Word Embedding Regularizationcited by 5
  15. 2022Data Augmentation for Biomedical Factoid Question Answeringcited by 5
  16. 2017Neural Question Answering at BioASQ 5Bcited by 4
  17. 2018CLINIQA: A Machine Intelligence Based Clinical Question Answering Systemcited by 4
  18. 2020Causal Perception in Question-Answering Systemscited by 4
  19. 2019Question Answering based Clinical Text Structuring Using Pre-trained Language Modelcited by 1
  20. 2022Science Checker: Extractive-Boolean Question Answering For Scientific Fact Checkingcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 1.

3 papers.

  1. 2024RealMedQA: A pilot biomedical question answering dataset containing realistic clinical questionscited by 5
  2. 2024Optimized Biomedical Question-Answering Services with LLM and Multi-BERT Integrationcited by 2
  3. 2025ECG-LLM -- training and evaluation of domain-specific large language models for electrocardiographycited by 0