UTSA-NLP at ArchEHR-QA 2025: Improving EHR Question Answering via Self-Consistency Prompting
On the map: LLM reasoning → Clinical and biomedical applications
Abstract (arXiv)
We describe our system for the ArchEHR-QA Shared Task on answering clinical questions using electronic health records (EHRs). Our approach uses large language models in two steps: first, to find sentences in the EHR relevant to a clinician's question, and second, to generate a short, citation-supported response based on those sentences. We use few-shot prompting, self-consistency, and thresholding to improve the sentence classification step to decide which sentences are essential. We compare several models and find that a smaller 8B model performs better than a larger 70B model for identifying relevant information. Our results show that accurate sentence selection is critical for generating high-quality responses and that self-consistency with thresholding helps make these decisions more reliable.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 5 of its 19 references that are in the map. First posted on arXiv 5 Jun 2025.
Builds on
5 of 19 references.
References · 5
- 2022Large Language Models Encode Clinical Knowledgecited by 1,830
- 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 325
- 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 118
- 2024UltraMedical: Building Specialized Generalists in Biomedicinecited by 67
- 2025A Dataset for Addressing Patient's Information Needs related to Clinical Course of Hospitalizationcited by 13
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Neural at ArchEHR-QA 2026: One Method Fits All: Unified Prompt Optimization for Clinical QA over EHRs
- 2026UIC-AIHealth4All at ArchEHR-QA 2026: Answer-First Evidence Grounding for Clinical Question Answering
- 2026Yale-DM-Lab at ArchEHR-QA 2026: Deterministic Grounding and Multi-Pass Evidence Alignment for EHR Question Answering
- 2025Neural at ArchEHR-QA 2025: Agentic Prompt Optimization for Evidence-Grounded Clinical Question Answering
- 2026Automating Clinical Information Retrieval from Finnish Electronic Health Records Using Large Language Models
- 2024Give me Some Hard Questions: Synthetic Data Generation for Clinical QA
- 2026Can Large Language Models Self-Correct in Medical Question Answering? An Exploratory Study
- 2026QU-NLP at ArchEHR-QA 2026: Two-Stage QLoRA Fine-Tuning of Qwen3-4B for Patient-Oriented Clinical Question Answering and Evidence Sentence Alignment