mlsanityData to 7 Oct 2026

Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence

arXiv 2412.01605 ↗First posted 2 Dec 2024cs.CL cs.AIcited by 15

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Clinical decision making (CDM) is a complex, dynamic process crucial to healthcare delivery, yet it remains a significant challenge for artificial intelligence systems. While Large Language Model (LLM)-based agents have been tested on general medical knowledge using licensing exams and knowledge question-answering tasks, their performance in the CDM in real-world scenarios is limited due to the lack of comprehensive testing datasets that mirror actual medical practice. To address this gap, we present MedChain, a dataset of 12,163 clinical cases that covers five key stages of clinical workflow. MedChain distinguishes itself from existing benchmarks with three key features of real-world clinical practice: personalization, interactivity, and sequentiality. Further, to tackle real-world CDM challenges, we also propose MedChain-Agent, an AI system that integrates a feedback mechanism and a MCase-RAG module to learn from previous cases and adapt its responses. MedChain-Agent demonstrates remarkable adaptability in gathering information dynamically and handling sequential clinical tasks, significantly outperforming existing approaches.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 68 references that are in the map. First posted on arXiv 2 Dec 2024.

Builds on

38 of 68 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments13 May

Other references · 37

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  10. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  11. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  12. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
25 more
  1. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  2. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  3. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
  4. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  5. 2023Towards Generalist Biomedical AIcited by 286
  6. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
  7. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  8. 2023CMB: A Comprehensive Medical Benchmark in Chinesecited by 113
  9. 2024Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questionscited by 106
  10. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 103
  11. 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
  12. 2024Towards Building Multilingual Language Model for Medicinecited by 91
  13. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
  14. 2024OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automationcited by 66
  15. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  16. 2024CoD, Towards an Interpretable Medical Agent using Chain of Diagnosiscited by 43
  17. 2019"The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learningcited by 39
  18. 2024Towards Evaluating and Building Versatile Large Language Models for Medicinecited by 38
  19. 2024Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generationcited by 30
  20. 2018Medical Exam Question Answering with Large-scale Reading Comprehensioncited by 29
  21. 2023MedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Modelscited by 29
  22. 2024MedBench: A Comprehensive, Standardized, and Reliable Benchmarking System for Evaluating Chinese Medical Large Language Modelscited by 21
  23. 2024A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Modelscited by 16
  24. 2024Almanac Copilot: Towards Autonomous Electronic Health Record Navigationcited by 9
  25. 2024CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarioscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 13.

15 papers.

Sort:

By area:

  1. 2026Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Makingcited by 10
  2. 2026Evolving Medical Imaging Agents via Experience-driven Self-skill Discoverycited by 10
  3. 2025WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysiscited by 7
  4. 2025Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluationcited by 5
  5. 2026MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosiscited by 4
  6. 2026ClinicalAgents: Multi-Agent Orchestration for Clinical Decision Making with Dual-Memorycited by 4
  7. 2026Benchmarking Egocentric Clinical Intent Understanding Capability for Medical Multimodal Large Language Modelscited by 1
  8. 2026Baichuan-M4: A Clinical-Grade Medical Agent System for Continuous Carecited by 1
  9. 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 1
  10. 2026Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agentscited by 1
  11. 2026CAREAgent: Clinical Agent with Structured Reasoning and Tool-Integrated for Order Generationcited by 0
  12. 2026MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Modelscited by 0
3 more
  1. 2026When Language Models Meet NeuroGraphs: Exploring Enhanced Agentic LLM Framework Towards Brain Network Analysiscited by 0
  2. 2026Adaptive Memory and Reflection Multi-Agent System for Medical Question Answeringcited by 0
  3. 2026From Given to Gathered Evidence: Agentic Learning for Longitudinal Medical Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?
  2. 2026Evaluating Large Language Models in Dynamic Clinical Decision-Making with Standardized Patient Cases
  3. 2025Building a Human-Verified Clinical Reasoning Dataset via a Human LLM Hybrid Pipeline for Trustworthy Medical AI
  4. 2025Reasoning LLMs in the Medical Domain: A Literature Survey
  5. 2026Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoning
  6. 2025MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
  7. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Models
  8. 2026Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation