mlsanityData to 7 Oct 2026

Simulated patient systems powered by large language model-based AI agents offer potential for transforming medical education

arXiv 2409.18924 ↗First posted 27 Sep 2024cs.CL cs.AIcited by 13

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Background: Simulated patient systems are important in medical education and research, providing safe, integrative training environments and supporting clinical decision making. Advances in artificial intelligence (AI), especially large language models (LLMs), can enhance simulated patients by replicating medical conditions and doctor patient interactions with high fidelity and at low cost, but effectiveness and trustworthiness remain open challenges. Methods: We developed AIPatient, a simulated patient system powered by LLM based AI agents. The system uses a retrieval augmented generation (RAG) framework with six task specific agents for complex reasoning. To improve realism, it is linked to the AIPatient knowledge graph built from de identified real patient data in the MIMIC III intensive care database. Results: We evaluated electronic health record (EHR) based medical question answering (QA), readability, robustness, stability, and user experience. AIPatient reached 94.15 percent QA accuracy when all six agents were enabled, outperforming versions with partial or no agent integration. The knowledge base achieved an F1 score of 0.89. Readability scores showed a median Flesch Reading Ease of 68.77 and a median Flesch Kincaid Grade of 6.4, indicating accessibility for most medical trainees and clinicians. Robustness and stability were supported by non significant variance in repeated trials (analysis of variance F value 0.61, p greater than 0.1; F value 0.78, p greater than 0.1). A user study with medical students showed that AIPatient provides high fidelity, usability, and educational value, comparable to or better than human simulated patients for history taking. Conclusions: LLM based simulated patient systems can deliver accurate, readable, and reliable medical encounters and show strong potential to transform medical education.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 9 of its 48 references that are in the map. First posted on arXiv 27 Sep 2024.

Builds on

9 of 48 references.

References · 9

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2024Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principlescited by 72
  3. 2023A Bibliometric Review of Large Language Models Research from 2017 to 2023cited by 56
  4. 2023Large Language Models to Identify Social Determinants of Health in Electronic Health Recordscited by 55
  5. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  6. 2024Towards Evaluating and Building Versatile Large Language Models for Medicinecited by 38
  7. 2024Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilitiescited by 22
  8. 2024Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysiscited by 9
  9. 2023CORAL: Expert-Curated medical Oncology Reports to Advance Language Model Inferencecited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 8.

13 papers.

Sort:

By area:

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  2. 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applicationscited by 10
  3. 2026MedDialBench: Benchmarking LLM Diagnostic Robustness under Parametric Adversarial Patient Behaviorscited by 4
  4. 2025Free Lunch for User Experience: Crowdsourcing Agents for Scalable User Studiescited by 1
  5. 2025Semantic Scheduling for LLM Inferencecited by 1
  6. 2026Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agentscited by 1
  7. 2025ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoningcited by 0
  8. 2026Human-AI Co-reasoning for Clinical Diagnosis with Evidence-Integrated Language Agentcited by 0
  9. 2026A Proactive Multi-Agent Dialogue Framework for Assessing Social Language Disorder Traits in Autismcited by 0
  10. 2026AgentCAT: Simulating Computerized Adaptive Testing via Multi-Agent Large Language Modelscited by 0
  11. 2026Knowledge-augmented Agentic AI for Mental Health Medication Information Seekingcited by 0
  12. 2026EmoPatient: An Emotion-Directed Patient Simulator for Realistic Palliative Care Communication Trainingcited by 0
1 more
  1. 2026Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator
  2. 2025AI Agents for Conversational Patient Triage: Preliminary Simulation-Based Evaluation with Real-World EHR Data
  3. 2023Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review
  4. 2025AutoMedic: An Automated Evaluation Framework for Clinical Conversational Agents with Medical Dataset Grounding
  5. 2024Towards Next-Generation Medical Agent: How o1 is Reshaping Decision-Making in Medical Scenarios
  6. 2024LLMs Can Simulate Standardized Patients via Agent Coevolution
  7. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?
  8. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents