# Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs

arXiv: https://arxiv.org/abs/2508.14817 · HTML: https://mlsanity.com/paper/2508.14817/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 20 Aug 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)
- Retrieval-augmented generation / [Medical and biomedical RAG](https://mlsanity.com/rag/medical-and-biomedical-rag/)

## Builds on (most cited first, 13 of 13)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization](https://mlsanity.com/paper/2309.07430/) (2023)
- [emrQA: A Large Corpus for Question Answering on Electronic Medical Records](https://mlsanity.com/paper/1809.00732/) (2018)
- [Almanac: Retrieval-Augmented Language Models for Clinical Medicine](https://mlsanity.com/paper/2303.01229/) (2023)
- [Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach](https://mlsanity.com/paper/2407.16833/) (2024)
- [MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records](https://mlsanity.com/paper/2308.14089/) (2023)
- [In Defense of RAG in the Era of Long-Context Language Models](https://mlsanity.com/paper/2409.01666/) (2024)
- [LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs -- No Silver Bullet for LC or RAG Routing](https://mlsanity.com/paper/2502.09977/) (2025)
- [DrugEHRQA: A Question Answering Dataset on Structured and Unstructured Electronic Health Records For Medicine Related Queries](https://mlsanity.com/paper/2205.01290/) (2022)
- [Lessons Learned on Information Retrieval in Electronic Health Records: A Comparison of Embedding Models and Pooling Strategies](https://mlsanity.com/paper/2409.15163/) (2024)
- [SNOBERT: A Benchmark for clinical notes entity linking in the SNOMED CT clinical terminology](https://mlsanity.com/paper/2405.16115/) (2024)

## Built on by (most cited first, 4 of 4)

- [TARSE: Test-Time Adaptation via Retrieval of Skills and Experience for Reasoning Agents](https://mlsanity.com/paper/2603.01241/) (2026)
- [Count-Based Approaches Remain Strong: A Benchmark Against Transformer and LLM Pipelines on Structured EHR](https://mlsanity.com/paper/2511.00782/) (2025)
- [Agentic clinical reasoning over longitudinal myeloma records: a retrospective evaluation against expert consensus](https://mlsanity.com/paper/2604.24473/) (2026)
- [ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning](https://mlsanity.com/paper/2606.02802/) (2026)

## Abstract (arXiv)

Objective: To evaluate whether retrieval-augmented generation (RAG) can serve as an efficient alternative to long-context prompting for clinical reasoning over electronic health records (EHRs). Methods: We defined three EHR-based tasks that are replicable across health systems and vary in reasoning complexity: 1) extracting imaging procedures (modality, date, and anatomic site), 2) generating timelines of therapeutic antibiotic use, and 3) identifying the key diagnoses for a hospitalization. Using real inpatient clinical notes from a US academic health system, we evaluated three large language models (GPT-5.4-mini, Mistral Medium 3, DeepSeek V3.1) with varying amounts of provided context, comparing targeted retrieval to using the most recent clinical notes. Results: For Imaging Procedures, RAG strongly outperformed recent-note inputs and exceeded long-context performance (by 0.17-9.83 F1 across all models) using fewer than 8K tokens. Similar benefits were observed for Antibiotic Timelines, where <8K of retrieved tokens matched long-context recent-notes performance (between -3.26 to +3.24 Jaccard). Error analysis revealed that missing information in the clinical notes--often due to inter-hospital transfers--limited performance to some extent. However, performance on the Diagnosis Generation task remains largely static across methods and models. Discussion: RAG demonstrated strong token efficiency across tasks, with the clearest and most consistent gains observed for imaging extraction and antibiotic timeline reconstruction. Diagnosis generation proved the most challenging task, suggesting ceiling effects imposed by documentation variability and evaluation constraints. Conclusion: Our results suggest that RAG remains a competitive and efficient approach for clinical tasks over large amounts of EHR, even as newer models become capable of handling increasingly longer amounts of text.