# MemoRAG: Boosting Long Context Processing with Global Memory-Enhanced Retrieval Augmentation

arXiv: https://arxiv.org/abs/2409.05591 · HTML: https://mlsanity.com/paper/2409.05591/

## Key facts

- Cited by 119 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 35 of its 53 references that are in the map.
- First posted on arXiv 9 Sep 2024.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 30 of 35)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://mlsanity.com/paper/2305.13245/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [MuSiQue: Multihop Questions via Single-hop Question Composition](https://mlsanity.com/paper/2108.00573/) (2021)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [Active Retrieval Augmented Generation](https://mlsanity.com/paper/2305.06983/) (2023)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression](https://mlsanity.com/paper/2310.06839/) (2023)
- [Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model](https://mlsanity.com/paper/1906.01749/) (2019)
- [∞Bench: Extending Long Context Evaluation Beyond 100K Tokens](https://mlsanity.com/paper/2402.13718/) (2024)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Effective Long-Context Scaling of Foundation Models](https://mlsanity.com/paper/2309.16039/) (2023)
- [Distilling Knowledge from Reader to Retriever for Question Answering](https://mlsanity.com/paper/2012.04584/) (2020)
- [RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation](https://mlsanity.com/paper/2404.00610/) (2024)
- [LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning](https://mlsanity.com/paper/2401.01325/) (2024)
- [A Survey on Long Text Modeling with Transformers](https://mlsanity.com/paper/2302.14502/) (2023)

## Built on by (most cited first, 30 of 119)

- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks](https://mlsanity.com/paper/2602.16313/) (2026)
- [Mem-α: Learning Memory Construction via Reinforcement Learning](https://mlsanity.com/paper/2509.25911/) (2025)
- [AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications](https://mlsanity.com/paper/2602.22769/) (2026)
- [MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents](https://mlsanity.com/paper/2601.03236/) (2026)
- [Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics](https://mlsanity.com/paper/2505.00675/) (2025)
- [MemVerse: Multimodal Memory for Lifelong Learning Agents](https://mlsanity.com/paper/2512.03627/) (2025)
- [From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents](https://mlsanity.com/paper/2505.19549/) (2025)
- [MiniRAG: Towards Extremely Simple Retrieval-Augmented Generation](https://mlsanity.com/paper/2501.06713/) (2025)
- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces](https://mlsanity.com/paper/2602.03442/) (2026)
- [Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foraging](https://mlsanity.com/paper/2505.09316/) (2025)
- [MLP Memory: A Retriever-Pretrained Memory for Large Language Models](https://mlsanity.com/paper/2508.01832/) (2025)
- [HiMem: Hierarchical Long-Term Memory for LLM Long-Horizon Agents](https://mlsanity.com/paper/2601.06377/) (2026)
- [Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Framework](https://mlsanity.com/paper/2603.11768/) (2026)
- [Toward Efficient Agents: Memory, Tool learning, and Planning](https://mlsanity.com/paper/2601.14192/) (2026)
- [AtomMem : Learnable Dynamic Agentic Memory with Atomic Memory Operation](https://mlsanity.com/paper/2601.08323/) (2026)
- [LastingBench: Defend Benchmarks Against Knowledge Leakage](https://mlsanity.com/paper/2506.21614/) (2025)
- [Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions](https://mlsanity.com/paper/2602.22680/) (2026)
- [PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments](https://mlsanity.com/paper/2603.23231/) (2026)
- [TOM-SWE: User Mental Modeling For Software Engineering Agents](https://mlsanity.com/paper/2510.21903/) (2025)
- [ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage](https://mlsanity.com/paper/2410.16848/) (2024)
- [R3Mem: Bridging Memory Retention and Retrieval via Reversible Compression](https://mlsanity.com/paper/2502.15957/) (2025)
- [Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory](https://mlsanity.com/paper/2603.04257/) (2026)
- [Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents](https://mlsanity.com/paper/2606.06090/) (2026)
- [Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval](https://mlsanity.com/paper/2509.21710/) (2025)
- [MemWeaver: A Hierarchical Memory from Textual Interactive Behaviors for Personalized Generation](https://mlsanity.com/paper/2510.07713/) (2025)
- [MemCast: Memory-Driven Time Series Forecasting with Experience-Conditioned Reasoning](https://mlsanity.com/paper/2602.03164/) (2026)

## Abstract (arXiv)

Processing long contexts presents a significant challenge for large language models (LLMs). While recent advancements allow LLMs to handle much longer contexts than before (e.g., 32K or 128K tokens), it is computationally expensive and can still be insufficient for many applications. Retrieval-Augmented Generation (RAG) is considered a promising strategy to address this problem. However, conventional RAG methods face inherent limitations because of two underlying requirements: 1) explicitly stated queries, and 2) well-structured knowledge. These conditions, however, do not hold in general long-context processing tasks. In this work, we propose MemoRAG, a novel RAG framework empowered by global memory-augmented retrieval. MemoRAG features a dual-system architecture. First, it employs a light but long-range system to create a global memory of the long context. Once a task is presented, it generates draft answers, providing useful clues for the retrieval tools to locate relevant information within the long context. Second, it leverages an expensive but expressive system, which generates the final answer based on the retrieved information. Building upon this fundamental framework, we realize the memory module in the form of KV compression, and reinforce its memorization and cluing capacity from the Generation quality's Feedback (a.k.a. RLGF). In our experiments, MemoRAG achieves superior performances across a variety of long-context evaluation tasks, not only complex scenarios where traditional RAG methods struggle, but also simpler ones where RAG is typically applied.