# LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs -- No Silver Bullet for LC or RAG Routing

arXiv: https://arxiv.org/abs/2502.09977 · HTML: https://mlsanity.com/paper/2502.09977/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation.
- Builds on 36 of its 43 references that are in the map.
- First posted on arXiv 14 Feb 2025.

## Placed in

- Retrieval-augmented generation / [RAG pipeline design and evaluation](https://mlsanity.com/rag/rag-pipeline-design-and-evaluation/)

## Builds on (most cited first, 30 of 36)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024)
- [Benchmarking Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2309.01431/) (2023)
- [A Survey on Deep Learning for Named Entity Recognition](https://mlsanity.com/paper/1812.09449/) (2018)
- [ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2311.09476/) (2023)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://mlsanity.com/paper/2407.19669/) (2024)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [QuALITY: Question Answering with Long Input Texts, Yes!](https://mlsanity.com/paper/2112.08608/) (2021)
- [LooGLE: Can Long-Context Language Models Understand Long Contexts?](https://mlsanity.com/paper/2311.04939/) (2023)
- [ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding](https://mlsanity.com/paper/2305.14196/) (2023)
- [Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA](https://mlsanity.com/paper/2406.17419/) (2024)
- [Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach](https://mlsanity.com/paper/2407.16833/) (2024)
- [Retrieval meets Long Context Large Language Models](https://mlsanity.com/paper/2310.03025/) (2023)
- [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://mlsanity.com/paper/2105.03654/) (2021)
- [Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More?](https://mlsanity.com/paper/2406.13121/) (2024)
- [Calibrating LLM-Based Evaluator](https://mlsanity.com/paper/2309.13308/) (2023)

## Built on by (most cited first, 29 of 29)

- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [WebSailor: Navigating Super-human Reasoning for Web Agent](https://mlsanity.com/paper/2507.02592/) (2025)
- [Tongyi DeepResearch Technical Report](https://mlsanity.com/paper/2510.24701/) (2025)
- [WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning](https://mlsanity.com/paper/2509.13305/) (2025)
- [Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs](https://mlsanity.com/paper/2509.21361/) (2025)
- [RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models](https://mlsanity.com/paper/2505.23052/) (2025)
- [Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Find](https://mlsanity.com/paper/2505.18148/) (2025)
- [PRELUDE: A Benchmark Designed to Require Global Comprehension and Reasoning over Long Contexts](https://mlsanity.com/paper/2508.09848/) (2025)
- [Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapters](https://mlsanity.com/paper/2511.17044/) (2025)
- [RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routing](https://mlsanity.com/paper/2602.00296/) (2026)
- [Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs](https://mlsanity.com/paper/2508.14817/) (2025)
- [PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization](https://mlsanity.com/paper/2507.16679/) (2025)
- [URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2603.19281/) (2026)
- [How Retrieved Context Shapes Internal Representations in RAG](https://mlsanity.com/paper/2602.20091/) (2026)
- [OmniBench-RAG: A Multi-Domain Evaluation Platform for Retrieval-Augmented Generation Tools](https://mlsanity.com/paper/2508.05650/) (2025)
- [Incentive-Aligned Multi-Source LLM Summaries](https://mlsanity.com/paper/2509.25184/) (2025)
- [NC-Bench: An LLM Benchmark for Evaluating Conversational Competence](https://mlsanity.com/paper/2601.06426/) (2026)
- [S3-Attention:Attention-Aligned Endogenous Retrieval for Memory-Bounded Long-Context Inference](https://mlsanity.com/paper/2601.17702/) (2026)
- [R3AG: Retriever Routing for Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.22849/) (2026)
- [Context Is Not Comprehension](https://mlsanity.com/paper/2506.04907/) (2025)
- [Agri-Query: A Case Study on RAG vs. Long-Context LLMs for Cross-Lingual Technical Question Answering](https://mlsanity.com/paper/2508.18093/) (2025)
- [Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding](https://mlsanity.com/paper/2509.21865/) (2025)
- [On the Sensitivity of Instruction-tuned LLMs to Harmful Sentences in Long Inputs](https://mlsanity.com/paper/2510.05864/) (2025)
- [Flash-Fusion: Enabling Expressive, Low-Latency Queries on IoT Sensor Streams with LLMs](https://mlsanity.com/paper/2511.11885/) (2025)
- [SagaScale: A Realistic, Scalable, and High-Quality Long-Context Benchmark Built from Full-Length Novels](https://mlsanity.com/paper/2601.09723/) (2025)
- [Human-Like Lifelong Memory: A Neuroscience-Grounded Architecture for Infinite Interaction](https://mlsanity.com/paper/2603.29023/) (2026)
- [Trace Only What You Need: Structure-Aware On-Demand Hypergraph Memory for Long-Document Question Answering](https://mlsanity.com/paper/2606.10921/) (2026)
- [MGAL: A Multilingual Granularity-Aware Long-Context Benchmark](https://mlsanity.com/paper/2608.20853/) (2026)
- [LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning](https://mlsanity.com/paper/2609.38137/) (2026)

## Abstract (arXiv)

Effectively incorporating external knowledge into Large Language Models (LLMs) is crucial for enhancing their capabilities and addressing real-world needs. Retrieval-Augmented Generation (RAG) offers an effective method for achieving this by retrieving the most relevant fragments into LLMs. However, the advancements in context window size for LLMs offer an alternative approach, raising the question of whether RAG remains necessary for effectively handling external knowledge. Several existing studies provide inconclusive comparisons between RAG and long-context (LC) LLMs, largely due to limitations in the benchmark designs. In this paper, we present LaRA, a novel benchmark specifically designed to rigorously compare RAG and LC LLMs. LaRA encompasses 2326 test cases across four practical QA task categories and three types of naturally occurring long texts. Through systematic evaluation of seven open-source and four proprietary LLMs, we find that the optimal choice between RAG and LC depends on a complex interplay of factors, including the model's parameter size, long-text capabilities, context length, task type, and the characteristics of the retrieved chunks. Our findings provide actionable guidelines for practitioners to effectively leverage both RAG and LC approaches in developing and deploying LLM applications. Our code and dataset is provided at: hrefhttps://github.com/Alibaba-NLP/LaRAhttps://github.com/Alibaba-NLP/LaRA.