# Language Agents for Hypothesis-driven Clinical Decision Making with Reinforcement Learning

arXiv: https://arxiv.org/abs/2506.13474 · HTML: https://mlsanity.com/paper/2506.13474/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)
- Deep reinforcement learning methods / [RL for clinical treatment and surgery](https://mlsanity.com/deeprl/rl-for-clinical-treatment-and-surgery/)

## Builds on (most cited first, 19 of 19)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [Towards Accurate Differential Diagnosis with Large Language Models](https://mlsanity.com/paper/2312.00164/) (2023)
- [Deep Reinforcement Learning for Cost-Effective Medical Diagnosis](https://mlsanity.com/paper/2302.10261/) (2023)
- [MAP: Evaluation and Multi-Agent Enhancement of Large Language Models for Inpatient Pathways](https://mlsanity.com/paper/2503.13205/) (2025)
- [ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic Assistance](https://mlsanity.com/paper/2402.13448/) (2024)
- [Med-PMC: Medical Personalized Multi-modal Consultation with a Proactive Ask-First-Observe-Next Paradigm](https://mlsanity.com/paper/2408.08693/) (2024)
- [Natural Language Programming in Medicine: Administering Evidence Based Clinical Workflows with Autonomous Agents Powered by Generative Large Language Models](https://mlsanity.com/paper/2401.02851/) (2024)

## Built on by (most cited first, 15 of 15)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning](https://mlsanity.com/paper/2510.04284/) (2025)
- [CaRT: Teaching LLM Agents to Know When They Know Enough](https://mlsanity.com/paper/2510.08517/) (2025)
- [Agentic Systems in Radiology: Design, Applications, Evaluation, and Challenges](https://mlsanity.com/paper/2510.09404/) (2025)
- [Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions](https://mlsanity.com/paper/2601.15353/) (2026)
- [ABRA: Agent Benchmark for Radiology Applications](https://mlsanity.com/paper/2605.11224/) (2026)
- [Joint Optimization of Reasoning and Dual-Memory for Self-Learning Diagnostic Agent](https://mlsanity.com/paper/2604.07269/) (2026)
- [CodeClinic: Evaluating Automation of Coding Skills for Clinical Reasoning Agents](https://mlsanity.com/paper/2605.09675/) (2026)
- [SAGEAgent: A Self-Evolving Agent for Cost-Aware Modality Acquisition in Multimodal Survival Prediction](https://mlsanity.com/paper/2607.09521/) (2026)
- [Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agents](https://mlsanity.com/paper/2609.09678/) (2026)
- [Learning Diagnostic Reasoning for Decision Support in Toxicology](https://mlsanity.com/paper/2603.29608/) (2026)
- [GPAgentBench-2K: Benchmarking Large Language Model Agents in Complex Clinical Action Space](https://mlsanity.com/paper/2608.30188/) (2026)
- [HPOQuest: A Rare-Disease Diagnostic Agent Using Active Phenotype Acquisition](https://mlsanity.com/paper/2609.18431/) (2026)
- [SCOPE-AD: Sequential cost-aware ordinal-belief planning with energy-based models for diagnostic agents](https://mlsanity.com/paper/2610.01278/) (2026)
- [Reinforcement Learning In Partially Verifiable Environments for Colon Cancer Treatment Planning](https://mlsanity.com/paper/2610.17426/) (2026)

## Abstract (arXiv)

Clinical decision-making is a dynamic, interactive, and cyclic process where doctors have to repeatedly decide on which clinical action to perform and consider newly uncovered information for diagnosis and treatment. Large Language Models (LLMs) have the potential to support clinicians in this process, however, most applications of LLMs in clinical decision support suffer from one of two limitations: Either they assume the unrealistic scenario of immediate availability of all patient information and do not model the interactive and iterative investigation process, or they restrict themselves to the limited "out-of-the-box" capabilities of large pre-trained models without performing task-specific training. In contrast to this, we propose to model clinical decision-making for diagnosis with a hypothesis-driven uncertainty-aware language agent, LA-CDM, that converges towards a diagnosis via repeatedly requesting and interpreting relevant tests. Using a hybrid training paradigm combining supervised and reinforcement learning, we train LA-CDM with three objectives targeting critical aspects of clinical decision-making: accurate hypothesis generation, hypothesis uncertainty estimation, and efficient decision-making. We evaluate our methodology on MIMIC-CDM, a real-world dataset covering four abdominal diseases containing various clinical tests and show the benefit of explicitly training clinical decision-making for increasing diagnostic performance and efficiency.