mlsanityData to 7 Oct 2026

Evaluation of AI Chatbots for Patient-Specific EHR Questions

arXiv 2306.02549 ↗First posted 5 Jun 2023cs.CL cs.AI cs.IRcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper investigates the use of artificial intelligence chatbots for patient-specific question answering (QA) from clinical notes using several large language model (LLM) based systems: ChatGPT (versions 3.5 and 4), Google Bard, and Claude. We evaluate the accuracy, relevance, comprehensiveness, and coherence of the answers generated by each model using a 5-point Likert scale on a set of patient-specific questions.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 7 of its 46 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

7 of 46 references.

References · 7

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  6. 2023A Bibliometric Review of Large Language Models Research from 2017 to 2023cited by 56
  7. 2018Annotating Electronic Medical Records for Question Answeringcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 6.

9 papers.

  1. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  2. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  3. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  4. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  5. 2024T-RAG: Lessons from the LLM Trenchescited by 23
  6. 2024PhenoFlow: A Human-LLM Driven Visual Analytics System for Exploring Large and Complex Stroke Datasetscited by 9
  7. 2023Question Answering for Electronic Health Records: A Scoping Review of datasets and modelscited by 7
  8. 2024How Reliable AI Chatbots are for Disease Prediction from Patient Complaints?cited by 5
  9. 2024Evaluating Search Engines and Large Language Models for Answering Health Questionscited by 5