mlsanityData to 7 Oct 2026

MEMORYLLM: Towards Self-Updatable Large Language Models

arXiv 2402.04624 ↗First posted 7 Feb 2024cs.CLcited by 89

Not on the map. Why?

Cited by 30 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Existing Large Language Models (LLMs) usually remain static after deployment, which might make it hard to inject new knowledge into the model. We aim to build models containing a considerable portion of self-updatable parameters, enabling the model to integrate new knowledge effectively and efficiently. To this end, we introduce MEMORYLLM, a model that comprises a transformer and a fixed-size memory pool within the latent space of the transformer. MEMORYLLM can self-update with text knowledge and memorize the knowledge injected earlier. Our evaluations demonstrate the ability of MEMORYLLM to effectively incorporate new knowledge, as evidenced by its performance on model editing benchmarks. Meanwhile, the model exhibits long-term information retention capacity, which is validated through our custom-designed evaluations and long-context benchmarks. MEMORYLLM also shows operational integrity without any sign of performance degradation even after nearly a million memory updates. Our code and model are open-sourced at https://github.com/wangyu-ustc/MemoryLLM.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Builds on 29 of its 33 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

29 of 33 references.

References · 29

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  7. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  8. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  9. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  10. 2021Editing Factual Knowledge in Language Modelscited by 548
  11. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 539
  12. 2021Fast Model Editing at Scalecited by 491
17 more
  1. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 425
  2. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  3. 2014Memory Networkscited by 354
  4. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  5. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  6. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  7. 2020Modifying Memories in Transformer Modelscited by 222
  8. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  9. 2022Recurrent Memory Transformercited by 197
  10. 2023Augmenting Language Models with Long-Term Memorycited by 186
  11. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  12. 2022A Length-Extrapolatable Transformercited by 123
  13. 2020Memory Transformercited by 74
  14. 2023RET-LLM: Towards a General Read-Write Memory for Large Language Modelscited by 64
  15. 2020Memformer: A Memory-Augmented Transformer for Sequence Modelingcited by 63
  16. 2022Global memory transformer for processing long documentscited by 4
  17. 2024UniMem: Towards a Unified View of Long-Context Large Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 23; 2026: 52.

89 papers.

Sort:

By area:

  1. 2024Titans: Learning to Memorize at Test Timecited by 303
  2. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  3. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
  4. 2025MIRIX: Multi-Agent Memory System for LLM-Based Agentscited by 160
  5. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 146
  6. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  7. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  8. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 99
  9. 2024WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Modelscited by 90
  10. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 90
  11. 2025M+: Extending MemoryLLM with Scalable Long-Term Memorycited by 51
  12. 2025Mem-α: Learning Memory Construction via Reinforcement Learningcited by 49
38 more
  1. 2024Online Adaptation of Language Models with a Memory of Amortized Contextscited by 35
  2. 2024MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memorycited by 35
  3. 2024In-Context Editing: Learning Knowledge from Self-Induced Distributionscited by 33
  4. 2025A Survey of Personalization: From RAG to Agentcited by 26
  5. 2025MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learningcited by 26
  6. 2026DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolutioncited by 23
  7. 2024Self-Updatable Large Language Models by Integrating Context into Model Parameterscited by 21
  8. 2024Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Modelscited by 20
  9. 2026LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleaguescited by 17
  10. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  11. 2026In-Place Test-Time Trainingcited by 15
  12. 2024Towards LifeSpan Cognitive Systemscited by 13
  13. 2024AI-native Memory: A Pathway from LLMs Towards AGIcited by 12
  14. 2026Evaluating Memory Structure in LLM Agentscited by 12
  15. 2025VTCBench: Can Vision-Language Models Understand Long Context with Vision-Text Compression?cited by 11
  16. 2026EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agentscited by 11
  17. 2026VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactionscited by 11
  18. 2025SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worldscited by 10
  19. 2025Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architecturescited by 9
  20. 2025LatentEvolve: Self-Evolving Test-Time Scaling in Latent Spacecited by 8
  21. 2025Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AIcited by 8
  22. 2026The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systemscited by 8
  23. 2026CL-bench Life: Can Language Models Learn from Real-Life Context?cited by 8
  24. 2025Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injectioncited by 7
  25. 2025Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retentioncited by 7
  26. 2026Position: Modular Memory is the Key to Continual Learning Agentscited by 7
  27. 2024CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Modelscited by 6
  28. 2025LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-Tuningcited by 6
  29. 2025TokMem: One-Token Procedural Memory for Large Language Modelscited by 5
  30. 2026EMBER: Efficient Memory via Budgeted Evidence Retention for Long-Horizon Agentscited by 5
  31. 2026TRUSTMEM: Learning Trustworthy Memory Consolidation for LLM Agents with Long-Term Memorycited by 5
  32. 2024Artificial Intelligence as the New Hacker: Developing Agents for Offensive Securitycited by 4
  33. 2024Deploying Foundation Model Powered Agent Services: A Surveycited by 4
  34. 2026HyMem: Hybrid Memory Architecture with Dynamic Retrieval Schedulingcited by 4
  35. 2026ParamMem: Augmenting Language Agents with Parametric Reflective Memorycited by 4
  36. 2025Generative AI Enhanced Financial Risk Management Information Retrievalcited by 3
  37. 2026G-MemLLM: Gated Latent Memory Augmentation for Long-Context Reasoning in Large Language Modelscited by 3
  38. 2026MemGym: a Long-Horizon Memory Environment for LLM Agentscited by 3

50 most cited of 89 All 89 (JSON)