mlsanityData to 14 Oct 2026

LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs

arXiv 2406.15319 ↗First posted 21 Jun 2024cs.CL cs.AIcited by 108

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

In traditional RAG framework, the basic retrieval units are normally short. The common retrievers like DPR normally work with 100-word Wikipedia paragraphs. Such a design forces the retriever to search over a large corpus to find the `needle' unit. In contrast, the readers only need to generate answers from the short retrieved units. The imbalanced `heavy' retriever and `light' reader design can lead to sub-optimal performance. The loss of contextual information in the short, chunked units may increase the likelihood of introducing hard negatives during the retrieval stage. Additionally, the reader might not fully leverage the capabilities of recent advancements in LLMs. In order to alleviate the imbalance, we propose a new framework LongRAG, consisting of a `long retriever' and a `long reader'. In the two Wikipedia-based datasets, NQ and HotpotQA, LongRAG processes the entire Wikipedia corpus into 4K-token units by grouping related documents. By increasing the unit size, we significantly reduce the total number of units. This greatly reduces the burden on the retriever, resulting in strong retrieval performance with only a few (less than 8) top units. Without requiring any training, LongRAG achieves an EM of 62.7% on NQ and 64.3% on HotpotQA, which are on par with the (fully-trained) SoTA model. Furthermore, we test on two non-Wikipedia-based datasets, Qasper and MultiFieldQA-en. LongRAG processes each individual document as a single (long) unit rather than chunking them into smaller units. By doing so, we achieve an F1 score of 25.9% on Qasper and 57.5% on MultiFieldQA-en. Our study offers insights into the future roadmap for combining RAG with long-context LLMs.

Key facts. Cited by 108 papers in the map (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 55 of its 70 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

55 of 70 references.

References · 55

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  8. 2017Billion-scale similarity search with GPUscited by 2,497
  9. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  11. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
38 more
  1. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  2. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  3. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  4. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  5. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  6. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  7. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  8. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  9. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 601
  10. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  11. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  12. 2023Augmented Language Models: a Surveycited by 403
  13. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  14. 2023Improving Text Embeddings with Large Language Modelscited by 303
  15. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 265
  16. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  17. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  18. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 162
  19. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  20. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 136
  21. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  22. 2019Deeper Text Understanding for IR with Contextual Neural Language Modelingcited by 113
  23. 2023PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Trainingcited by 111
  24. 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 104
  25. 2023Improving Language Models via Plug-and-Play Retrieval Feedbackcited by 87
  26. 2019Hierarchical Graph Network for Multi-hop Question Answeringcited by 83
  27. 2022Structured Prompting: Scaling In-Context Learning to 1,000 Examplescited by 81
  28. 2022Parallel Context Windows for Large Language Modelscited by 81
  29. 2021KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answeringcited by 67
  30. 2019Cognitive Graph for Multi-Hop Reading Comprehension at Scalecited by 65
  31. 2024Training-Free Long-Context Scaling of Large Language Modelscited by 62
  32. 2024LongEmbed: Extending Embedding Models for Long Context Retrievalcited by 48
  33. 2022Tevatron: An Efficient and Flexible Toolkit for Dense Retrievalcited by 47
  34. 2021UnitedQA: A Hybrid Approach for Open Domain Question Answeringcited by 42
  35. 2020Differentiable Reasoning over a Virtual Knowledge Basecited by 39
  36. 2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
  37. 2021Adaptive Information Seeking for Open-Domain Question Answeringcited by 29
  38. 2024Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERTcited by 21

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 23; 2025: 45; 2026: 40.

108 papers.

Sort:

By area:

  1. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  2. 2025ReasonIR: Training Retrievers for Reasoning Taskscited by 77
  3. 2025ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agentscited by 66
  4. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
  5. 2024EXAONE 3.5: Series of Large Language Models for Real-world Use Casescited by 53
  6. 2026MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agentscited by 52
  7. 2025Test-Time Learning for Large Language Modelscited by 50
  8. 2025Chain-of-Retrieval Augmented Generationcited by 49
  9. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 41
  10. 2024Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reductioncited by 39
  11. 2025MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughlycited by 29
  12. 2025How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmarkcited by 28
38 more
  1. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontierscited by 27
  2. 2025NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodescited by 26
  3. 2024Compute Or Load KV Cache? Why Not Both?cited by 25
  4. 2024A Survey on Large Language Models for Communication, Network, and Service Management: Application Insights, Challenges, and Future Directionscited by 25
  5. 2025Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answeringcited by 22
  6. 2025UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularitiescited by 20
  7. 2025RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discoverycited by 19
  8. 2024FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAGcited by 18
  9. 2024UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generationcited by 16
  10. 2025Hierarchical Document Refinement for Long-context Retrieval-augmented Generationcited by 16
  11. 2025Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compressioncited by 16
  12. 2024Towards LifeSpan Cognitive Systemscited by 13
  13. 2025Co-Sight: Enhancing LLM-Based Agents via Conflict-Aware Meta-Verification and Trustworthy Reasoning with Structured Factscited by 13
  14. 2026ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generationcited by 13
  15. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  16. 2024Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Modelscited by 9
  17. 2024Length-Induced Embedding Collapse in PLM-based Modelscited by 9
  18. 2025SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinementcited by 9
  19. 2025Enhancing Retrieval Augmented Generation with Hierarchical Text Segmentation Chunkingcited by 8
  20. 2026Retrieval--Reasoning Processes for Multi-hop Question Answering: A Four-Axis Design Framework and Empirical Trendscited by 8
  21. 2025DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generationcited by 7
  22. 2026MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Modelscited by 7
  23. 2024Squid: Long Context as a New Modality for Energy-Efficient On-Device Language Modelscited by 6
  24. 2024Unified Multimodal Interleaved Document Representation for Retrievalcited by 6
  25. 2024Unleashing Multi-Hop Reasoning Potential in Large Language Models through Repetition of Misordered Contextcited by 6
  26. 2024LIFT: Improving Long Context Understanding Through Long Input Fine-Tuningcited by 6
  27. 2025LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-Tuningcited by 6
  28. 2025Cluster-based Adaptive Retrieval: Dynamic Context Selection for RAG Applicationscited by 6
  29. 2025Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapterscited by 6
  30. 2026HAGE: Harnessing Agentic Memory via RL-Driven Weighted Graph Evolutioncited by 6
  31. 2026ReCA: Multi-Shot Long Video Extrapolation via Recursive Context Allocationcited by 6
  32. 2024UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrieverscited by 5
  33. 2025DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillationcited by 5
  34. 2024SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detectioncited by 4
  35. 2024Bootstrap Your Own Context Lengthcited by 4
  36. 2025Enhancing Large Language Models with Neurosymbolic Reasoning for Multilingual Taskscited by 4
  37. 2025How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking Modelscited by 4
  38. 2025Metadata-Driven Retrieval-Augmented Generation for Financial Question Answeringcited by 4

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Long Context vs. RAG for LLMs: An Evaluation and Revisits
  2. 2024Long Context RAG Performance of Large Language Models
  3. 2024MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation
  4. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering
  5. 2024Long2RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall
  6. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
  7. 2025SemRAG: Semantic Knowledge-Augmented RAG for Improved Question-Answering
  8. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG