Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering
Not on the map. Why?
Abstract (arXiv)
We describe the Yale-DM-Lab system for the ArchEHR-QA 2026 shared task. The task studies patient-authored questions about hospitalization records and contains four subtasks (ST): clinician-interpreted question reformulation, evidence sentence identification, answer generation, and evidence-answer alignment. ST1 uses a dual-model pipeline with Claude Sonnet 4 and GPT-4o to reformulate patient questions into clinician-interpreted questions. ST2-ST4 rely on Azure-hosted model ensembles (o3, GPT-5.2, GPT-5.1, and DeepSeek-R1) combined with few-shot prompting and voting strategies. Our experiments show three main findings. First, model diversity and ensemble voting consistently improve performance compared to single-model baselines. Second, the full clinician answer paragraph is provided as additional prompt context for evidence alignment. Third, results on the development set show that alignment accuracy is mainly limited by reasoning. The best scores on the development set reach 88.81 micro F1 on ST4, 65.72 macro F1 on ST2, 34.01 on ST3, and 33.05 on ST1.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
16 of 29 references.
References · 16
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
- 2022Large Language Models Encode Clinical Knowledgecited by 1,807
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
- 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
- 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
- 2019Neural Text Summarization: A Critical Evaluationcited by 189
4 more
- 2023Do We Still Need Clinical Language Models?cited by 97
- 2024EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summariescited by 33
- 2025A Dataset for Addressing Patient's Information Needs related to Clinical Course of Hospitalizationcited by 13
- 2026TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generationcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026UIC-AIHealth4All at ArchEHR-QA 2026: Answer-First Evidence Grounding for Clinical Question Answering
- 2026Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs
- 2025UTSA-NLP at ArchEHR-QA 2025: Improving EHR Question Answering via Self-Consistency Prompting
- 2025MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answering
- 2025Neural at ArchEHR-QA 2025: Agentic Prompt Optimization for Evidence-Grounded Clinical Question Answering
- 2025ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning
- 2023Aligning Large Language Models for Clinical Tasks
- 2026EHRNavigator: A Multi-Agent System for Patient-Level Clinical Question Answering over Heterogeneous Electronic Health Records