mlsanityData to 14 Oct 2026

MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models

arXiv 2606.24155 ↗First posted 23 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing medical AI benchmarks lack process visibility, atomic skill evaluation, and integrated hallucination detection. We introduce MedBench v5, a redesigned benchmark for clinical multimodal models (language, vision-language, and agent systems) that moves from static QA to dynamic, process-oriented evaluation. MedBench v5 features: (1) a dual-dimensional framework combining Clinical Cognitive Responsiveness (13 sub-dimensions) and Medical Atomic Skills (4 agent environments), covering 63 tasks; (2) three switchable information-flow stressors (omission, contradiction, evidence delay) for factorized degradation analysis; (3) a dynamic process audit protocol with five reasoning nodes that produces model-specific failure fingerprints; (4) hallucination propagation monitoring across initiation, propagation, anchoring, and contradiction interaction-capturing silent hallucination. Experiments on frontier models show that strong overall task performance does not guarantee process stability: stressors mainly disrupt contradiction detection, diagnosis updating, hallucination propagation, and contradiction-based self-correction, while final evidence grounding can remain superficially stable. MedBench v5 provides a unified infrastructure for capability profiling, controllable stress testing, process auditing, and hallucination trajectory analysis in clinical AI evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 64 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

28 of 64 references.

References · 28

  1. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  2. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  3. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 369
  4. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
  5. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 157
  6. 2024MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoningcited by 154
  7. 2025Deep Research Agents: A Systematic Examination And Roadmapcited by 128
  8. 2021CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmarkcited by 101
  9. 2025Medical Hallucinations in Foundation Models and Their Impact on Healthcarecited by 93
  10. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 91
  11. 2024EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Recordscited by 78
  12. 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Modelscited by 57
16 more
  1. 2025MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Modelscited by 34
  2. 2024ClinicalLab: Aligning Agents for Multi-Departmental Clinical Diagnostics in the Real Worldcited by 28
  3. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Modelscited by 22
  4. 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMscited by 21
  5. 2024Medchain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequencecited by 17
  6. 2025From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculationscited by 11
  7. 2025Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Modelscited by 9
  8. 2025LLM Sensitivity Evaluation Framework for Clinical Diagnosiscited by 7
  9. 2026CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligencecited by 5
  10. 2026MedDialBench: Benchmarking LLM Diagnostic Robustness under Parametric Adversarial Patient Behaviorscited by 4
  11. 2026ClinEnv: An Interactive Multi-Stage Long Horizon EHR Environment for Agentscited by 4
  12. 2026Strong Reasoning Isn't Enough: Evaluating Evidence Elicitation in Interactive Diagnosiscited by 3
  13. 2026Beyond Idealized Patients: Evaluating LLMs under Challenging Patient Behaviors in Medical Consultationscited by 3
  14. 2026MedConceal: A Benchmark for Clinical Hidden-Concern Reasoning Under Partial Observabilitycited by 2
  15. 2026ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoningcited by 2
  16. 2026DDX-TRACE: A Benchmark for Medical Diagnostic Trajectories in VLMscited by 1

Built on by

Not cited yet.