mlsanityData to 14 Oct 2026

sebis at ArchEHR-QA 2026: How Much Can You Do Locally? Evaluating Grounded EHR QA on a Single Notebook

arXiv 2603.13962 ↗First posted 14 Mar 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Clinical question answering over electronic health records (EHRs) can help clinicians and patients access relevant medical information more efficiently. However, many recent approaches rely on large cloud-based models, which are difficult to deploy in clinical environments due to privacy constraints and computational requirements. In this work, we investigate how far grounded EHR question answering can be pushed when restricted to a single notebook. We participate in all four subtasks of the ArchEHR-QA 2026 shared task and evaluate several approaches designed to run on commodity hardware. All experiments are conducted locally without external APIs or cloud infrastructure. Our results show that such systems can achieve competitive performance on the shared task leaderboards. In particular, our submissions perform above average in two subtasks, and we observe that smaller models can approach the performance of much larger systems when properly configured. These findings suggest that privacy-preserving EHR QA systems running fully locally are feasible with current models and commodity hardware. The source code is available at https://github.com/ibrahimey/ArchEHR-QA-2026.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 56 references that are in the map. First posted on arXiv 14 Mar 2026.

Builds on

16 of 56 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  7. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 493
  10. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 288
  11. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 185
  12. 2018emrQA: A Large Corpus for Question Answering on Electronic Medical Recordscited by 135
4 more
  1. 2023ACI-BENCH: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generationcited by 118
  2. 2025Prompt Repetition Improves Non-Reasoning LLMscited by 27
  3. 2025A Dataset for Addressing Patient's Information Needs related to Clinical Course of Hospitalizationcited by 13
  4. 2023Question Answering for Electronic Health Records: A Scoping Review of datasets and modelscited by 7

Built on by

Not cited yet.