# CP-Env: Evaluating Large Language Models on Clinical Pathways in a Controllable Hospital Environment

arXiv: https://arxiv.org/abs/2512.10206 · HTML: https://mlsanity.com/paper/2512.10206/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 87 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Builds on (most cited first, 26 of 26)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Social Simulacra: Creating Populated Prototypes for Social Computing Systems](https://mlsanity.com/paper/2208.04024/) (2022)
- [LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals](https://mlsanity.com/paper/2411.10109/) (2024)
- [AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments](https://mlsanity.com/paper/2405.07960/) (2024)
- [War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars](https://mlsanity.com/paper/2311.17227/) (2023)
- [Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questions](https://mlsanity.com/paper/2402.18060/) (2024)
- [AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator](https://mlsanity.com/paper/2402.09742/) (2024)
- [Unveiling the Truth and Facilitating Change: Towards Agent-based Large-scale Social Movement Simulation](https://mlsanity.com/paper/2402.16333/) (2024)
- [MedCalc-Bench: Evaluating Large Language Models for Medical Calculations](https://mlsanity.com/paper/2406.12036/) (2024)
- [SocioVerse: A World Model for Social Simulation Powered by LLM Agents and A Pool of 10 Million Real-World Users](https://mlsanity.com/paper/2504.10157/) (2025)
- [AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation](https://mlsanity.com/paper/2403.02959/) (2024)
- [Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions](https://mlsanity.com/paper/2503.22678/) (2025)
- [MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Education](https://mlsanity.com/paper/2404.06711/) (2024)
- [Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases](https://mlsanity.com/paper/2503.04691/) (2025)
- [MedAgentBoard: Benchmarking Multi-Agent Collaboration with Conventional Methods for Diverse Medical Tasks](https://mlsanity.com/paper/2505.12371/) (2025)
- [CitySim: Modeling Urban Behaviors and City Dynamics with Large-Scale LLM-Driven Agent Simulation](https://mlsanity.com/paper/2506.21805/) (2025)
- [Can Large Language Models abstract Medical Coded Language?](https://mlsanity.com/paper/2403.10822/) (2024)
- [InternBootcamp: Boosting LLM Reasoning with Verifiable Task Scaling](https://mlsanity.com/paper/2508.08636/) (2025)
- [Don't Just Fine-tune the Agent, Tune the Environment](https://mlsanity.com/paper/2510.10197/) (2025)
- [Ready Jurist One: Benchmarking Language Agents for Legal Intelligence in Dynamic Environments](https://mlsanity.com/paper/2507.04037/) (2025)
- [MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Calling](https://mlsanity.com/paper/2410.13610/) (2024)
- [MMXU: A Multi-Modal and Multi-X-ray Understanding Dataset for Disease Progression](https://mlsanity.com/paper/2502.11651/) (2025)
- [Chinese Court Simulation with LLM-Based Agent System](https://mlsanity.com/paper/2508.17322/) (2025)

## Built on by (most cited first, 4 of 4)

- [EHRWorld: A Patient-Centric Medical World Model for Long-Horizon Clinical Trajectories](https://mlsanity.com/paper/2602.03569/) (2026)
- [HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents](https://mlsanity.com/paper/2606.31179/) (2026)
- [Dual Attention Guided Defense Against Malicious Edits](https://mlsanity.com/paper/2512.14333/) (2025)
- [MorphologyFM: A Foundation Model for Morphology-Aware Representation Learning from ECG and Pulse Oximetry Waveforms](https://mlsanity.com/paper/2607.09749/) (2026)

## Abstract (arXiv)

Medical care follows complex clinical pathways that extend beyond isolated physician-patient encounters, emphasizing decision-making and transitions between different stages. Current benchmarks focusing on static exams or isolated dialogues inadequately evaluate large language models (LLMs) in dynamic clinical scenarios. We introduce CP-Env, a controllable agentic hospital environment designed to evaluate LLMs across end-to-end clinical pathways. CP-Env simulates a hospital ecosystem with patient and physician agents, constructing scenarios ranging from triage and specialist consultation to diagnostic testing and multidisciplinary team meetings for agent interaction. Following real hospital adaptive flow of healthcare, it enables branching, long-horizon task execution. We propose a three-tiered evaluation framework encompassing Clinical Efficacy, Process Competency, and Professional Ethics. Results reveal that most models struggle with pathway complexity, exhibiting hallucinations and losing critical diagnostic details. Interestingly, excessive reasoning steps can sometimes prove counterproductive, while top models tend to exhibit reduced tool dependency through internalized knowledge. CP-Env advances medical AI agents development through comprehensive end-to-end clinical evaluation. We provide the benchmark and evaluation tools for further research and development at https://github.com/SPIRAL-MED/CP_ENV.