# Stronger Baselines for Retrieval-Augmented Generation with Long-Context Language Models

arXiv: https://arxiv.org/abs/2506.03989 · HTML: https://mlsanity.com/paper/2506.03989/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 13 of its 14 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 13 of 13)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [∞Bench: Extending Long Context Evaluation Beyond 100K Tokens](https://mlsanity.com/paper/2402.13718/) (2024)
- [QuALITY: Question Answering with Long Input Texts, Yes!](https://mlsanity.com/paper/2112.08608/) (2021)
- [A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts](https://mlsanity.com/paper/2402.09727/) (2024)
- [Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading](https://mlsanity.com/paper/2310.05029/) (2023)
- [Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach](https://mlsanity.com/paper/2407.16833/) (2024)
- [PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents](https://mlsanity.com/paper/2305.14564/) (2023)
- [In Defense of RAG in the Era of Long-Context Language Models](https://mlsanity.com/paper/2409.01666/) (2024)
- [Embedding And Clustering Your Data Can Improve Contrastive Pretraining](https://mlsanity.com/paper/2407.18887/) (2024)

## Built on by (most cited first, 9 of 9)

- [RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routing](https://mlsanity.com/paper/2602.00296/) (2026)
- [Presentation, Not Mechanism: A Render Confound in Deprecation-Aware Memory Evaluation](https://mlsanity.com/paper/2607.16019/) (2026)
- [Injecting Knowledge from Social Science Journals to Improve Indonesian Cultural Understanding by LLMs](https://mlsanity.com/paper/2601.12921/) (2026)
- [M-RAG: Semantic Key-Value Indexing for Retrieval-Augmented Generation](https://mlsanity.com/paper/2603.26667/) (2026)
- [Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.08046/) (2026)
- [StateRAG: Typed State Contracts for Complex Retrieval-Augmented Generation](https://mlsanity.com/paper/2605.25379/) (2026)
- [Imprint: Online Memory Compression for Long-Horizon Egocentric QA](https://mlsanity.com/paper/2607.00696/) (2026)
- [RENDER: Controlling Reader-Facing Evidence in LLM Memory Evaluation](https://mlsanity.com/paper/2608.23568/) (2026)
- [The Copy Ceiling: An Input-Exposure Control for Ontology-Grounded Generation over Curated Corpora](https://mlsanity.com/paper/2609.24885/) (2026)

## Abstract (arXiv)

With the rise of long-context language models (LMs) capable of processing tens of thousands of tokens in a single context window, do multi-stage retrieval-augmented generation (RAG) pipelines still offer measurable benefits over simpler, single-stage approaches? To assess this question, we conduct a controlled evaluation for QA tasks under systematically scaled token budgets, comparing two recent multi-stage pipelines, ReadAgent and RAPTOR, against three baselines, including DOS RAG (Document's Original Structure RAG), a simple retrieve-then-read method that preserves original passage order. Despite its straightforward design, DOS RAG consistently matches or outperforms more intricate methods on multiple long-context QA benchmarks. We trace this strength to a combination of maintaining source fidelity and document structure, prioritizing recall within effective context windows, and favoring simplicity over added pipeline complexity. We recommend establishing DOS RAG as a simple yet strong baseline for future RAG evaluations, paired with state-of-the-art embedding and language models, and benchmarked under matched token budgets, to ensure that added pipeline complexity is justified by clear performance gains as models continue to improve.