ClinicalBench: Stress-Testing Assertion-Aware Retrieval for Cross-Admission Clinical QA on MIMIC-IV
Not on the map. Why?
Abstract (arXiv)
Reasoning benchmarks measure clinical performance on clean inputs. We evaluate the step before reasoning: retrieval over real EHR notes, where negation, temporality, and family-versus-patient attribution can flip a correct answer to a wrong one. EpiKG carries an assertion label and a temporality tag with every fact in a patient knowledge graph, then routes retrieval by question intent. ClinicalBench is a 400-question test over 43 MIMIC-IV patients across 9 assertion-sensitive categories. A 7-condition ablation tests each piece of EpiKG across six LLMs (Claude Opus 4.6, GPT-OSS 20B, MedGemma 27B, Gemma 4 31B, MedGemma 1.5 4B, Qwen 3.5 35B). Three physicians blindly adjudicated 100 paired items. The author-blind primary endpoint, leave-author-out paired exact McNemar on 50 unanimous-strict items rated by two external physicians, yields +22.0 percentage points (95 percent Newcombe CI [+5.1, +31.5], p=0.0192). The architectural novelty, intent-aware KG-RAG over a Contriever dense-RAG baseline (C2b to C4g_kw on the change-excluded n=362 endpoint), is +8.84 percentage points (paired McNemar p=1.79e-3); +12.43 percentage points under oracle intent. Sensitivities agree directionally: three-rater physician majority +24.0 percentage points (subject to single-author circularity); deterministic keyword reproducibility proxy +39.5 percentage points. Across the six models, the gain shrinks as the LLM-alone baseline rises (beta=-1.123, r=-0.921, p=0.009). With n=6 this looks more like regression to the mean than encoding substituting for model size. Physician adjudication identified 56 percent of auto-generated reference answers as defective, a methodological finding indicating that NLP-pipeline clinical-QA benchmarks require physician adjudication to be usable. ClinicalBench, the frozen evaluator, three-rater adjudication data, and the EpiKG output stack are publicly released.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 17 of its 45 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
17 of 45 references.
References · 17
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
- 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
- 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 400
- 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
- 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
- 2024Capabilities of Gemini Models in Medicinecited by 237
- 2018emrQA: A Large Corpus for Question Answering on Electronic Medical Recordscited by 135
- 2024Medical Graph RAG: Towards Safe Medical Large Language Model via Graph Retrieval-Augmented Generationcited by 90
- 2017NegBio: a high-performance tool for negation and uncertainty detection in radiology reportscited by 79
- 2024Towards Conversational Diagnostic AIcited by 79
- 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
5 more
- 2023Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Studycited by 30
- 2026EHR-RAG: Bridging Long-Horizon Structured Electronic Health Records and Large Language Models via Enhanced Retrieval-Augmented Generationcited by 6
- 2024AutoRD: An Automatic and End-to-End System for Rare Disease Knowledge Graph Construction Based on Ontologies-enhanced Large Language Modelscited by 3
- 2025Beyond Negation Detection: Comprehensive Assertion Detection Models for Clinical NLPcited by 1
- 2025Unlocking Electronic Health Records: A Hybrid Graph RAG Approach to Safe Clinical AI for Patient QAcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026GraphWalker: Patient Analogy Meets Information Gain for Clinical Reasoning with Large Language Models
- 2025Beyond Long Context: When Semantics Matter More than Tokens
- 2025Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs
- 2026MED-COPILOT: A Medical Assistant Powered by GraphRAG and Similar Patient Case Retrieval
- 2026PubMed Reasoner: Dynamic Reasoning-based Retrieval for Evidence-Grounded Biomedical Question Answering
- 2026ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues
- 2025Experience Retrieval-Augmentation with Electronic Health Records Enables Accurate Discharge QA
- 2025ASTRID -- An Automated and Scalable TRIaD for the Evaluation of RAG-based Clinical Question Answering Systems