mlsanityData to 14 Oct 2026

LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering

arXiv 2410.18050 ↗First posted 23 Oct 2024cs.CLcited by 49

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Long-Context Question Answering (LCQA), a challenging task, aims to reason over long-context documents to yield accurate answers to questions. Existing long-context Large Language Models (LLMs) for LCQA often struggle with the "lost in the middle" issue. Retrieval-Augmented Generation (RAG) mitigates this issue by providing external factual evidence. However, its chunking strategy disrupts the global long-context information, and its low-quality retrieval in long contexts hinders LLMs from identifying effective factual details due to substantial noise. To this end, we propose LongRAG, a general, dual-perspective, and robust LLM-based RAG system paradigm for LCQA to enhance RAG's understanding of complex long-context knowledge (i.e., global information and factual details). We design LongRAG as a plug-and-play paradigm, facilitating adaptation to various domains and LLMs. Extensive experiments on three multi-hop datasets demonstrate that LongRAG significantly outperforms long-context LLMs (up by 6.94%), advanced RAG (up by 6.16%), and Vanilla RAG (up by 17.25%). Furthermore, we conduct quantitative ablation studies and multi-dimensional analyses, highlighting the effectiveness of the system's components and fine-tuning strategies. Data and code are available at https://github.com/QingFei1/LongRAG.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 36 of its 54 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

36 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection17 Oct

Other references · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  8. 2023A Survey of Large Language Modelscited by 2,576
  9. 2017Billion-scale similarity search with GPUscited by 2,497
  10. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  11. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
23 more
  1. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  2. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  3. 2023LIMA: Less Is More for Alignmentcited by 1,100
  4. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  5. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  6. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  7. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  8. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  9. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  10. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  11. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 313
  12. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
  13. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  14. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 195
  15. 2022Rethinking with Retrieval: Faithful Large Language Model Inferencecited by 145
  16. 2024LongAlign: A Recipe for Long Context Alignment of Large Language Modelscited by 106
  17. 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
  18. 2023Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Taskscited by 59
  19. 2023A Survey on Long Text Modeling with Transformerscited by 47
  20. 2023Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Rankingcited by 44
  21. 2024Grounding Language Model with Chunking-Free In-Context Retrievalcited by 30
  22. 2021Long Context Question Answering via Supervised Contrastive Learningcited by 17
  23. 2024LongHeads: Multi-Head Attention is Secretly a Long Context Processorcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 30; 2026: 18.

49 papers.

Sort:

By area:

  1. 2025Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learningcited by 39
  2. 2025R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learningcited by 34
  3. 2025PersonaX: A Recommendation Agent Oriented User Modeling Framework for Long Behavior Sequencecited by 33
  4. 2025Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunitiescited by 31
  5. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  6. 2025U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystackcited by 20
  7. 2025Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foragingcited by 20
  8. 2025VoiceAgentBench: Are Voice Assistants ready for agentic tasks?cited by 19
  9. 2025TeleMem: Building Long-Term and Multimodal Memory for Agentic AIcited by 14
  10. 2026Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answeringcited by 14
  11. 2025IC-Cache: Efficient Large Language Model Serving via In-context Cachingcited by 13
  12. 2025Self-Taught Agentic Long Context Understandingcited by 10
37 more
  1. 2025What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMscited by 8
  2. 2026Retrieval--Reasoning Processes for Multi-hop Question Answering: A Four-Axis Design Framework and Empirical Trendscited by 8
  3. 2025Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retentioncited by 7
  4. 2025Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAGcited by 7
  5. 2026DeepRead: Document Structure-Aware Reasoning to Enhance Agentic Searchcited by 7
  6. 2026HYPIC: Accelerating Hybrid-Attention LLM Serving with Position-Independent Cachingcited by 7
  7. 2025HawkBench: Investigating Resilience of RAG Methods on Stratified Information-Seeking Taskscited by 6
  8. 2025VersionRAG: Version-Aware Retrieval-Augmented Generation for Evolving Documentscited by 6
  9. 2026JADE: Bridging the Strategic-Operational Gap in Dynamic Agentic RAGcited by 6
  10. 2025Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generationcited by 5
  11. 2025Quantifying the Impact of Structured Output Format on Large Language Models through Causal Inferencecited by 5
  12. 2026DF-RAG: Query-Aware Diversity for Retrieval-Augmented Generationcited by 5
  13. 2026OmniRAG-Agent: Agentic Omnimodal Reasoning for Low-Resource Long Audio-Video Question Answeringcited by 5
  14. 2025Metadata-Driven Retrieval-Augmented Generation for Financial Question Answeringcited by 4
  15. 2025R-Debater: Retrieval-Augmented Debate Generation through Argumentative Memorycited by 4
  16. 2025Extract, Match, and Score: An Evaluation Paradigm for Long Question-context-answer Triplets in Financial Analysiscited by 3
  17. 2025From Chat Logs to Collective Insights: Aggregative Question Answeringcited by 3
  18. 2024Mixture-of-PageRanks: Replacing Long-Context with Real-Time, Sparse GraphRAGcited by 2
  19. 2025MacRAG: Compress, Slice, and Scale-up for Multi-Scale Adaptive Context RAGcited by 2
  20. 2026SF-RAG: Structure-Fidelity Retrieval-Augmented Generation for Academic Question Answeringcited by 2
  21. 2026DySCO: Dynamic Attention-Scaling Decoding for Long-Context Language Modelscited by 2
  22. 2026Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signalscited by 2
  23. 2025POLYRAG: Integrating Polyviews into Retrieval-Augmented Generation for Medical Applicationscited by 1
  24. 2025Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaborationcited by 1
  25. 2025AuditAgent: Expert-Guided Multi-Agent Reasoning for Cross-Document Fraudulent Evidence Discoverycited by 1
  26. 2026Long-Context Long-Form Question Answering for Legal Domaincited by 1
  27. 2026Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelinescited by 1
  28. 2026InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gaincited by 1
  29. 2026Evidence-State Rewards for Long-Context Reasoningcited by 1
  30. 2025ICA-RAG: Information Completeness Guided Adaptive Retrieval-Augmented Generation for Disease Diagnosiscited by 0
  31. 2025FB-RAG: Improving RAG with Forward and Backward Lookupcited by 0
  32. 2025GOSU: Retrieval-Augmented Generation with Global-Level Optimized Semantic Unit-Centric Frameworkcited by 0
  33. 2025On the Sensitivity of Instruction-tuned LLMs to Harmful Sentences in Long Inputscited by 0
  34. 2026EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answeringcited by 0
  35. 2026CTRAG: An In-Context Retrieval-based Framework for Automated Compliance Checking using LLMscited by 0
  36. 2026W-RAG: Source-Aware Retrieval for Enterprise Document Generation from Heterogeneous Knowledge Basescited by 0
  37. 2026Beyond Linear Context: Graph-Guided Evidence Navigation for Long-Novel Reasoning with a Local 9B Language Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024In Defense of RAG in the Era of Long-Context Language Models
  2. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
  3. 2025On the Influence of Context Size and Model Choice in Retrieval-Augmented Generation Systems
  4. 2024ALR2: A Retrieve-then-Reason Framework for Long-context Question Answering
  5. 2024Long2RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall
  6. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering
  7. 2024Long Context vs. RAG for LLMs: An Evaluation and Revisits
  8. 2024MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation