mlsanityData to 14 Oct 2026

MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation

arXiv 2409.05591 ↗First posted 9 Sep 2024cs.CL cs.AIcited by 119

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Processing long contexts presents a significant challenge for large language models (LLMs). While recent advancements allow LLMs to handle much longer contexts than before (e.g., 32K or 128K tokens), it is computationally expensive and can still be insufficient for many applications. Retrieval-Augmented Generation (RAG) is considered a promising strategy to address this problem. However, conventional RAG methods face inherent limitations because of two underlying requirements: 1) explicitly stated queries, and 2) well-structured knowledge. These conditions, however, do not hold in general long-context processing tasks. In this work, we propose MemoRAG, a novel RAG framework empowered by global memory-augmented retrieval. MemoRAG features a dual-system architecture. First, it employs a light but long-range system to create a global memory of the long context. Once a task is presented, it generates draft answers, providing useful clues for the retrieval tools to locate relevant information within the long context. Second, it leverages an expensive but expressive system, which generates the final answer based on the retrieved information. Building upon this fundamental framework, we realize the memory module in the form of KV compression, and reinforce its memorization and cluing capacity from the Generation quality's Feedback (a.k.a. RLGF). In our experiments, MemoRAG achieves superior performances across a variety of long-context evaluation tasks, not only complex scenarios where traditional RAG methods struggle, but also simpler ones where RAG is typically applied.

Key facts. Cited by 119 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 35 of its 53 references that are in the map. First posted on arXiv 9 Sep 2024.

Builds on

35 of 53 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Active Retrieval Augmented Generation11 May
  2. 2023In-Context Retrieval-Augmented Language Models31 Jan
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May

Other references · 32

  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2017Billion-scale similarity search with GPUscited by 2,497
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  7. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  8. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  11. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  12. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
20 more
  1. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  2. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  3. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  4. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  5. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  6. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  7. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  8. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  9. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
  10. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  11. 2023Effective Long-Context Scaling of Foundation Modelscited by 303
  12. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  13. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 147
  14. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 136
  15. 2023A Survey on Long Text Modeling with Transformerscited by 47
  16. 2024Grounding Language Model with Chunking-Free In-Context Retrievalcited by 30
  17. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  18. 2024Are Long-LLMs A Necessity For Long-Context Tasks?cited by 22
  19. 2021Rethinking Search: Making Domain Experts out of Dilettantescited by 18
  20. 2024Extending Llama-3's Context Ten-Fold Overnightcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 39; 2026: 76.

119 papers.

Sort:

By area:

  1. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  3. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  4. 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 84
  5. 2025Mem-α: Learning Memory Construction via Reinforcement Learningcited by 55
  6. 2026AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applicationscited by 53
  7. 2026MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agentscited by 52
  8. 2025Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topicscited by 36
  9. 2025MemVerse: Multimodal Memory for Lifelong Learning Agentscited by 35
  10. 2025From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agentscited by 29
  11. 2025MiniRAG: Towards Extremely Simple Retrieval-Augmented Generationcited by 28
  12. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunitiescited by 28
38 more
  1. 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfacescited by 24
  2. 2025Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foragingcited by 20
  3. 2025MLP Memory: A Retriever-Pretrained Memory for Large Language Modelscited by 20
  4. 2026HiMem: Hierarchical Long-Term Memory for LLM Long-Horizon Agentscited by 18
  5. 2026Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Frameworkcited by 18
  6. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  7. 2026AtomMem : Learnable Dynamic Agentic Memory with Atomic Memory Operationcited by 15
  8. 2025LastingBench: Defend Benchmarks Against Knowledge Leakagecited by 14
  9. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 14
  10. 2026PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environmentscited by 14
  11. 2025TOM-SWE: User Mental Modeling For Software Engineering Agentscited by 13
  12. 2024ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coveragecited by 12
  13. 2025R3Mem: Bridging Memory Retention and Retrieval via Reversible Compressioncited by 12
  14. 2026Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memorycited by 12
  15. 2026Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agentscited by 11
  16. 2025Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrievalcited by 10
  17. 2025MemWeaver: A Hierarchical Memory from Textual Interactive Behaviors for Personalized Generationcited by 9
  18. 2026MemCast: Memory-Driven Time Series Forecasting with Experience-Conditioned Reasoningcited by 9
  19. 2026MAP-Graph: Provenance-Aware Shared Memory for Multi-Agent Workflowscited by 9
  20. 2025Memory in Large Language Models: Mechanisms, Evaluation and Evolutioncited by 8
  21. 2025Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AIcited by 8
  22. 2026Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoningcited by 8
  23. 2026IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inferencecited by 7
  24. 2025HawkBench: Investigating Resilience of RAG Methods on Stratified Information-Seeking Taskscited by 6
  25. 2025From Ranking to Selection: A Simple but Efficient Dynamic Passage Selector for Retrieval Augmented Generationcited by 6
  26. 2025OpsAgent: An Evolving Multi-agent System for Incident Management in Microservicescited by 6
  27. 2026AMA: Adaptive Memory via Multi-Agent Collaborationcited by 6
  28. 2026SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generationcited by 6
  29. 2026HAGE: Harnessing Agentic Memory via RL-Driven Weighted Graph Evolutioncited by 6
  30. 2026Infini Memory: Maintainable Topic Documents for Long-Term LLM Agent Memorycited by 6
  31. 2025Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decodingcited by 5
  32. 2025Mindscape-Aware Retrieval Augmented Generation for Improved Long Context Understandingcited by 5
  33. 2026SwiftMem: Fast Agentic Memory via Query-aware Indexingcited by 5
  34. 2026PruneRAG: Confidence-Guided Query Decomposition Trees for Efficient Retrieval-Augmented Generationcited by 5
  35. 2026Complementary RL: Towards Efficient Experience-Driven Agent Learningcited by 5
  36. 2025HGMEM: Hypergraph-based Working Memory to Improve Multi-step RAG for Long-Context Complex Relational Modelingcited by 4
  37. 2026RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehensioncited by 4
  38. 2026Hierarchical Memory Orchestration for Personalized Persistent Agentscited by 4

50 most cited of 119 All 119 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  2. 2025Does RAG Really Perform Bad For Long-Context Processing?
  3. 2024Toward Optimal Search and Retrieval for RAG
  4. 2025Tuning LLMs by RAG Principles: Towards LLM-native Memory
  5. 2024Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey
  6. 2024Enhancing Long Context Performance in LLMs Through Inner Loop Query Mechanism
  7. 2024LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering
  8. 2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach