# MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models

arXiv: https://arxiv.org/abs/2606.24155 · HTML: https://mlsanity.com/paper/2606.24155/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 64 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 28 of 28)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering](https://mlsanity.com/paper/2203.14371/) (2022)
- [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [Med-HALT: Medical Domain Hallucination Test for Large Language Models](https://mlsanity.com/paper/2307.15343/) (2023)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [Deep Research Agents: A Systematic Examination And Roadmap](https://mlsanity.com/paper/2506.18096/) (2025)
- [CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2106.08087/) (2021)
- [Medical Hallucinations in Foundation Models and Their Impact on Healthcare](https://mlsanity.com/paper/2503.05777/) (2025)
- [Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Dataset](https://mlsanity.com/paper/2306.03030/) (2023)
- [EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Records](https://mlsanity.com/paper/2401.07128/) (2024)
- [Detecting and Evaluating Medical Hallucinations in Large Vision Language Models](https://mlsanity.com/paper/2406.10185/) (2024)
- [MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models](https://mlsanity.com/paper/2502.14302/) (2025)
- [ClinicalLab: Aligning Agents for Multi-Departmental Clinical Diagnostics in the Real World](https://mlsanity.com/paper/2406.13890/) (2024)
- [Automating Expert-Level Medical Reasoning Evaluation of Large Language Models](https://mlsanity.com/paper/2507.07988/) (2025)
- [OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs](https://mlsanity.com/paper/2601.01592/) (2026)
- [Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence](https://mlsanity.com/paper/2412.01605/) (2024)
- [From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations](https://mlsanity.com/paper/2509.16584/) (2025)
- [Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models](https://mlsanity.com/paper/2510.10278/) (2025)
- [LLM Sensitivity Evaluation Framework for Clinical Diagnosis](https://mlsanity.com/paper/2504.13475/) (2025)
- [CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence](https://mlsanity.com/paper/2605.12882/) (2026)
- [MedDialBench: Benchmarking LLM Diagnostic Robustness under Parametric Adversarial Patient Behaviors](https://mlsanity.com/paper/2604.06846/) (2026)
- [ClinEnv: An Interactive Multi-Stage Long Horizon EHR Environment for Agents](https://mlsanity.com/paper/2606.02568/) (2026)
- [Strong Reasoning Isn't Enough: Evaluating Evidence Elicitation in Interactive Diagnosis](https://mlsanity.com/paper/2601.19773/) (2026)
- [Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultations](https://mlsanity.com/paper/2603.29373/) (2026)
- [MedConceal: A Benchmark for Clinical Hidden-Concern Reasoning Under Partial Observability](https://mlsanity.com/paper/2604.08788/) (2026)
- [ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning](https://mlsanity.com/paper/2606.14697/) (2026)
- [DDX-TRACE: A Benchmark for Medical Diagnostic Trajectories in VLMs](https://mlsanity.com/paper/2605.23629/) (2026)

## Abstract (arXiv)

Existing medical AI benchmarks lack process visibility, atomic skill evaluation, and integrated hallucination detection. We introduce MedBench v5, a redesigned benchmark for clinical multimodal models (language, vision-language, and agent systems) that moves from static QA to dynamic, process-oriented evaluation. MedBench v5 features: (1) a dual-dimensional framework combining Clinical Cognitive Responsiveness (13 sub-dimensions) and Medical Atomic Skills (4 agent environments), covering 63 tasks; (2) three switchable information-flow stressors (omission, contradiction, evidence delay) for factorized degradation analysis; (3) a dynamic process audit protocol with five reasoning nodes that produces model-specific failure fingerprints; (4) hallucination propagation monitoring across initiation, propagation, anchoring, and contradiction interaction-capturing silent hallucination. Experiments on frontier models show that strong overall task performance does not guarantee process stability: stressors mainly disrupt contradiction detection, diagnosis updating, hallucination propagation, and contradiction-based self-correction, while final evidence grounding can remain superficially stable. MedBench v5 provides a unified infrastructure for capability profiling, controllable stress testing, process auditing, and hallucination trajectory analysis in clinical AI evaluation.