mlsanityData to 7 Oct 2026

Long Context vs. RAG for LLMs: An Evaluation and Revisits

arXiv 2501.01880 ↗First posted 27 Dec 2024cs.CLcited by 24

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Extending context windows (i.e., Long Context, LC) and using retrievers to selectively access relevant information (i.e., Retrieval-Augmented Generation, RAG) are the two main strategies to enable LLMs to incorporate extremely long external contexts. This paper revisits recent studies on this topic, highlighting their key insights and discrepancies. We then provide a more comprehensive evaluation by filtering out questions answerable without external context, identifying the most effective retrieval methods, and expanding the datasets. We show that LC generally outperforms RAG in question-answering benchmarks, especially for Wikipedia-based questions. Summarization-based retrieval performs comparably to LC, while chunk-based retrieval lags behind. However, RAG has advantages in dialogue-based and general question queries. These insights underscore the trade-offs between RAG and LC strategies, offering guidance for future optimization of LLMs with external knowledge sources. We also provide an in-depth discussion on this topic, highlighting the overlooked importance of context relevance in existing studies.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. First posted on arXiv 27 Dec 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 10.

24 papers.

Sort:

By area:

  1. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 118
  2. 2025The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMscited by 28
  3. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  4. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  5. 2025Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-kcited by 19
  6. 2025FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generationcited by 17
  7. 2025GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?cited by 16
  8. 2025RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decodingcited by 14
  9. 2026Retrieval--Reasoning Processes for Multi-hop Question Answering: A Four-Axis Design Framework and Empirical Trendscited by 8
  10. 2025AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Sciencecited by 7
  11. 2026EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agentscited by 7
  12. 2026VimRAG: Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graphcited by 5
12 more
  1. 2025T2-RAGBench: Text-and-Table Benchmark for Evaluating Retrieval-Augmented Generationcited by 3
  2. 2025OkraLong: A Flexible Retrieval-Augmented Framework for Long-Text Query Processingcited by 2
  3. 2026Exploring Fine-Tuning for In-Context Retrieval and Efficient KV-Caching in Long-Context Language Modelscited by 1
  4. 2026P2RAG: Efficient Privacy-Preserving RAG Service Supporting Arbitrary Top-k Retrievalcited by 1
  5. 2026The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Contextcited by 1
  6. 2025Dynamic Long Short-Term Memory Based Memory Storage For Long Horizon LLM Interactioncited by 0
  7. 2025Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answeringcited by 0
  8. 2025Mesh RAG: Retrieval Augmentation for Autoregressive Mesh Generationcited by 0
  9. 2026Beyond the Needle's Illusion: Decoupled Evaluation of Evidence Access and Use under Semantic Interference at 326M-Token Scalecited by 0
  10. 2026Retrieval Is Not Enough: Why Organizational AI Needs Epistemic Infrastructurecited by 0
  11. 2026FD-RAG: Federated Dual-System Retrieval-Augmented Generationcited by 0
  12. 2026Effects of Transcript Compression on LLM-based Medical Misinformation Detection in Japanese YouTube Videoscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024In Defense of RAG in the Era of Long-Context Language Models
  2. 2025LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs -- No Silver Bullet for LC or RAG Routing
  3. 2025On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems
  4. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
  5. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering
  6. 2024Long2RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall
  7. 2024Context Awareness Gate For Retrieval Augmented Generation
  8. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs