mlsanityData to 14 Oct 2026

MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory

arXiv 2404.11672 ↗First posted 17 Apr 2024cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

While current large language models (LLMs) perform well on many knowledge-related tasks, they are limited by relying on their parameters as an implicit storage mechanism. As a result, they struggle with memorizing rare events and with updating their memory as facts change over time. In addition, the uninterpretable nature of parametric memory makes it challenging to prevent hallucination. Model editing and augmenting LLMs with parameters specialized for memory are only partial solutions. In this paper, we introduce MemLLM, a novel method of enhancing LLMs by integrating a structured and explicit read-and-write memory module. MemLLM tackles the aforementioned challenges by enabling dynamic interaction with the memory and improving the LLM's capabilities in using stored knowledge. Our experiments indicate that MemLLM enhances the LLM's performance and interpretability, in language modeling in general and knowledge-intensive tasks in particular. We see MemLLM as an important step towards making LLMs more grounded and factual through memory augmentation. The project repository is publicly available at https://github.com/amodaresi/MemLLM

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 58 of its 71 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

58 of 71 references.

References · 58

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  8. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  9. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  10. 2020Longformer: The Long-Document Transformercited by 3,413
  11. 2023Mistral 7Bcited by 3,164
  12. 2022Locating and Editing Factual Associations in GPTcited by 2,691
38 more
  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  2. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  5. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  6. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  7. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  8. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  9. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
  10. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  11. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  13. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  14. 2021Editing Factual Knowledge in Language Modelscited by 553
  15. 2021Fast Model Editing at Scalecited by 498
  16. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  17. 2022Memory-Based Model Editing at Scalecited by 447
  18. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  19. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  20. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  21. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  22. 2022RealTime QA: What's the Answer Right Now?cited by 244
  23. 2019DocRED: A Large-Scale Document-Level Relation Extraction Datasetcited by 239
  24. 2022Memorizing Transformerscited by 226
  25. 2024Repeat After Me: Transformers are Better than State Space Models at Copyingcited by 216
  26. 2022Recurrent Memory Transformercited by 203
  27. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 197
  28. 2023Augmenting Language Models with Long-Term Memorycited by 189
  29. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  30. 2020Editable Neural Networkscited by 155
  31. 2023ChatDB: Augmenting LLMs with Databases as Their Symbolic Memorycited by 140
  32. 2023Walking Down the Memory Maze: Beyond Context Limit through Interactive Readingcited by 133
  33. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 119
  34. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  35. 2023SCM: Enhancing Large Language Model with Self-Controlled Memory Frameworkcited by 98
  36. 2024MEMORYLLM: Towards Self-Updatable Large Language Modelscited by 93
  37. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  38. 2023RecurrentGPT: Interactive Generation of (Arbitrarily) Long Textcited by 84

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 13; 2026: 17.

35 papers.

Sort:

By area:

  1. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 237
  2. 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systemscited by 122
  3. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontierscited by 65
  4. 2025M+: Extending MemoryLLM with Scalable Long-Term Memorycited by 54
  5. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 44
  6. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agentscited by 36
  7. 2025O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agentscited by 32
  8. 2024Logically Consistent Language Models via Neuro-Symbolic Integrationcited by 30
  9. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 25
  10. 2025ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memorycited by 24
  11. 2025CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehensioncited by 23
  12. 2024Uncovering Logit Suppression Vulnerabilities in LLM Safety Alignmentcited by 18
23 more
  1. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  2. 2025ENGRAM: Effective, Lightweight Memory Orchestration for Conversational Agentscited by 15
  3. 2026Evaluating Memory Structure in LLM Agentscited by 12
  4. 2024Problem Solving Through Human-AI Preference-Based Cooperationcited by 11
  5. 2025Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architecturescited by 9
  6. 2025ExLLM: Experience-Enhanced LLM Optimization for Molecular Design and Beyondcited by 8
  7. 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agentscited by 7
  8. 2024Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generationcited by 5
  9. 2026AutoMem: Automated Learning of Memory as a Cognitive Skillcited by 5
  10. 2024OrthoDoc: Multimodal Large Language Model for Assisting Diagnosis in Computed Tomographycited by 3
  11. 2025Memorization and Knowledge Injection in Gated LLMscited by 3
  12. 2026Higher-Order Knowledge Representations for Agentic Scientific Reasoningcited by 3
  13. 2025The Paradox of Poetic Intent in Back-Translation: Evaluating the Quality of Large Language Models in Chinese Translationcited by 2
  14. 2026Distilling Feedback into Memory-as-a-Toolcited by 2
  15. 2026AdMem: Advanced Memory for Task-solving Agentscited by 1
  16. 2026Selective Forgetting: A Graph-Based Memory Framework for Long-Term LLM Agentscited by 1
  17. 2026MemX: A Local-First Long-Term Memory System for AI Assistantscited by 0
  18. 2026LANTERN: Layered Archival and Temporal Episodic Retrieval Network for Long-Context LLM Conversationscited by 0
  19. 2026Rosetta Memory: Adaptive Memory for Cross-LLM Agentscited by 0
  20. 2026Co-LMLM: Continuous-Query Limited Memory Language Modelscited by 0
  21. 2026LoKiFormer: Locality-aware Attention with Decoupled Knowledge Memory for Efficient Large Language Model Pretrainingcited by 0
  22. 2026Transferable End-to-End Optimization for Indirect Long-Term Memory Poisoning in LLM Agentscited by 0
  23. 2026Eternal Sunshine of the Spotless Mind: Systematically Erasing LLM's Memoriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Models
  2. 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models
  3. 2024LLM Inference Unveiled: Survey and Roofline Model Insights
  4. 2024Memory3: Language Modeling with Explicit Memory
  5. 2025MemLoRA: Distilling Expert Adapters for On-Device Memory Systems
  6. 2024MM-LLMs: Recent Advances in MultiModal Large Language Models
  7. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts
  8. 2026Towards In-Parameter Memory Augmentation for Large Language Models