mlsanityData to 14 Oct 2026

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models

arXiv 2502.14802 ↗First posted 20 Feb 2025cs.CL cs.AIcited by 231

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language models (LLMs), retrieval-augmented generation (RAG) has become the dominant way to introduce new information. However, its reliance on vector retrieval hinders its ability to mimic the dynamic and interconnected nature of human long-term memory. Recent RAG approaches augment vector embeddings with various structures like knowledge graphs to address some of these gaps, namely sense-making and associativity. However, their performance on more basic factual memory tasks drops considerably below standard RAG. We address this unintended deterioration and propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in HippoRAG and enhances it with deeper passage integration and more effective online use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the-art embedding model while also exhibiting superior factual knowledge and sense-making memory capabilities. This work paves the way for non-parametric continual learning for LLMs. Code and data are available at https://github.com/OSU-NLP-Group/HippoRAG.

Key facts. Cited by 231 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 28 of its 44 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

28 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories20 Dec

Other references · 27

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  4. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  5. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  6. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  7. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  8. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  9. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 532
  10. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  11. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  12. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
15 more
  1. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  2. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  3. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
  4. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  5. 2024Generative Representational Instruction Tuningcited by 233
  6. 2023Walking Down the Memory Maze: Beyond Context Limit through Interactive Readingcited by 133
  7. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 126
  8. 2024Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsalcited by 118
  9. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 115
  10. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 103
  11. 2024BM25S: Orders of magnitude faster lexical search via eager sparse scoringcited by 88
  12. 2023Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmarkcited by 62
  13. 2023CITB: A Benchmark for Continual Instruction Tuningcited by 33
  14. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answeringcited by 28
  15. 2023Carpe Diem: On the Evaluation of World Knowledge in Lifelong Language Modelscited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 69; 2026: 162.

231 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
  3. 2025MemOS: A Memory OS for AI Systemcited by 138
  4. 2025WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoningcited by 53
  5. 2025Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learningcited by 52
  6. 2026STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?cited by 50
  7. 2025LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corporacited by 49
  8. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 48
  9. 2025Retrieval-Augmented Generation with Hierarchical Knowledgecited by 47
  10. 2025Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topicscited by 36
  11. 2026PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agentscited by 32
  12. 2025RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learningcited by 30
38 more
  1. 2025From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agentscited by 29
  2. 2025SGMem: Sentence Graph Memory for Long-Term Conversational Agentscited by 29
  3. 2025Synergizing RAG and Reasoning: A Systematic Reviewcited by 27
  4. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 25
  5. 2025PropRAG: Guiding Retrieval with Beam Search over Proposition Pathscited by 23
  6. 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
  7. 2026REMem: Reasoning with Episodic Memory in Language Agentcited by 23
  8. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
  9. 2025A Simple Yet Strong Baseline for Long-Term Conversational Memory of LLM Agentscited by 21
  10. 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 18
  11. 2025Pre-Storage Reasoning for Episodic Memory: Shifting Inference Burden to Memory for Personalized Dialoguecited by 18
  12. 2026HiMem: Hierarchical Long-Term Memory for LLM Long-Horizon Agentscited by 18
  13. 2026According to Me: Long-Term Personalized Referential Memory QAcited by 14
  14. 2026HyperMem: Hypergraph Memory for Long-Term Conversationscited by 14
  15. 2025RAG Meets Temporal Graphs: Time-Sensitive Modeling and Retrieval for Evolving Knowledgecited by 13
  16. 2026VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactionscited by 12
  17. 2026Choosing How to Remember: Adaptive Memory Structures for LLM Agentscited by 11
  18. 2025Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrievalcited by 10
  19. 2025UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Modelscited by 9
  20. 2025G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledgecited by 9
  21. 2026AgentLongBench: A Controllable Long Benchmark For Long-Contexts Agents via Environment Rolloutscited by 9
  22. 2026All-Mem: Agentic Lifelong Memory via Dynamic Topology Evolutioncited by 9
  23. 2025EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notescited by 8
  24. 2025ReMindRAG: Low-Cost LLM-Guided Knowledge Graph Traversal for Efficient RAGcited by 8
  25. 2025Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoningcited by 8
  26. 2026Dynamic Long Context Reasoning over Compressed Memory via End-to-End Reinforcement Learningcited by 8
  27. 2026MemORAI: Memory Organization and Retrieval via Adaptive Graph Intelligence for LLM Conversational Agentscited by 8
  28. 2026Reliable Post-Retrieval Assembly for Agent Memory: Separating Evidence Extraction from Policy Executioncited by 8
  29. 2025AutoGraph-R1: End-to-End Reinforcement Learning for Knowledge Graph Constructioncited by 7
  30. 2025RGMem: Renormalization Group-inspired Memory Evolution for Language Agentscited by 7
  31. 2025TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Frameworkcited by 7
  32. 2025AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMscited by 7
  33. 2026GraphSearch: Agentic Search-Augmented Reasoning for Zero-Shot Graph Learningcited by 7
  34. 2026Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloadscited by 7
  35. 2025Incorporating Legal Structure in Retrieval-Augmented Generation: A Case Study on Copyright Fair Usecited by 6
  36. 2025Beyond Chunks and Graphs: Retrieval-Augmented Generation through Triplet-Driven Thinkingcited by 6
  37. 2026MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokenscited by 6
  38. 2025Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentationcited by 5

50 most cited of 231 All 231 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
  2. 2024The Power of Noise: Redefining Retrieval for RAG Systems
  3. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
  5. 2025Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing
  6. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  7. 2026Self-Aware Vector Embeddings for Retrieval-Augmented Generation: A Neuroscience-Inspired Framework for Temporal, Confidence-Weighted, and Relational Knowledge
  8. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models