mlsanityData to 14 Oct 2026

M+: Extending MemoryLLM with Scalable Long-Term Memory

arXiv 2502.00592 ↗First posted 1 Feb 2025cs.CLcited by 54

Not on the map. Why?

Cited by 29 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Equipping large language models (LLMs) with latent-space memory has attracted increasing attention as they can extend the context window of existing language models. However, retaining information from the distant past remains a challenge. For example, MemoryLLM (Wang et al., 2024a), as a representative work with latent-space memory, compresses past information into hidden states across all layers, forming a memory pool of 1B parameters. While effective for sequence lengths up to 16k tokens, it struggles to retain knowledge beyond 20k tokens. In this work, we address this limitation by introducing M+, a memory-augmented model based on MemoryLLM that significantly enhances long-term information retention. M+ integrates a long-term memory mechanism with a co-trained retriever, dynamically retrieving relevant information during text generation. We evaluate M+ on diverse benchmarks, including long-context understanding and knowledge retention tasks. Experimental results show that M+ significantly outperforms MemoryLLM and recent strong baselines, extending knowledge retention from under 20k to over 160k tokens with similar GPU memory overhead. We open-source our code at https://github.com/wangyu-ustc/MemoryLLM

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

30 of 39 references.

References · 30

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  3. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  4. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  6. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  7. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  8. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  9. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
  10. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
  11. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 281
  12. 2022Memorizing Transformerscited by 226
18 more
  1. 2021A Survey of Human-in-the-loop for Machine Learningcited by 215
  2. 2022Recurrent Memory Transformercited by 203
  3. 2023In-context Autoencoder for Context Compression in a Large Language Modelcited by 199
  4. 2023Augmenting Language Models with Long-Term Memorycited by 189
  5. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 186
  6. 2023ChatDB: Augmenting LLMs with Databases as Their Symbolic Memorycited by 140
  7. 2023Scaling Transformer to 1M tokens and beyond with RMTcited by 98
  8. 2024MEMORYLLM: Towards Self-Updatable Large Language Modelscited by 93
  9. 2023RecurrentGPT: Interactive Generation of (Arbitrarily) Long Textcited by 84
  10. 2024Memory3: Language Modeling with Explicit Memorycited by 40
  11. 2024MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memorycited by 35
  12. 2024Who's Who: Large Language Models Meet Knowledge Conflicts in Practicecited by 21
  13. 2024Large Scale Knowledge Washingcited by 16
  14. 2024Towards LifeSpan Cognitive Systemscited by 13
  15. 2023MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answeringcited by 12
  16. 2024Tiny Transformers Excel at Sentence Compressioncited by 2
  17. 2024An Enhanced Text Compression Approach Using Transformer-based Language Modelscited by 2
  18. 2024MemoryPrompt: A Light Wrapper to Improve Context Tracking in Pre-trained Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 35.

54 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
  3. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
  4. 2025MIRIX: Multi-Agent Memory System for LLM-Based Agentscited by 173
  5. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  6. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 109
  7. 2025Mem-α: Learning Memory Construction via Reinforcement Learningcited by 55
  8. 2025MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learningcited by 27
  9. 2026DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolutioncited by 26
  10. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  11. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 18
  12. 2025WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidancecited by 16
38 more
  1. 2025Towards General Continuous Memory for Vision-Language Modelscited by 15
  2. 2026According to Me: Long-Term Personalized Referential Memory QAcited by 14
  3. 2026δ-mem: Efficient Online Memory for Large Language Modelscited by 13
  4. 2026Evaluating Memory Structure in LLM Agentscited by 12
  5. 2026Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memorycited by 12
  6. 2025M3HF: Multi-agent Reinforcement Learning from Multi-phase Human Feedback of Mixed Qualitycited by 11
  7. 2025Artificial Hippocampus Networks for Efficient Long-Context Modelingcited by 11
  8. 2025Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architecturescited by 9
  9. 2025LatentEvolve: Self-Evolving Test-Time Scaling in Latent Spacecited by 9
  10. 2025Beyond Pipelines: A Survey of the Paradigm Shift toward Model-Native Agentic AIcited by 8
  11. 2025Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injectioncited by 7
  12. 2026TRUSTMEM: Learning Trustworthy Memory Consolidation for LLM Agents with Long-Term Memorycited by 6
  13. 2026MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Searchcited by 5
  14. 2026G-MemLLM: Gated Latent Memory Augmentation for Long-Context Reasoning in Large Language Modelscited by 4
  15. 2026ElasticMem: Latent Memory as a Learnable Resource for LLM Agentscited by 4
  16. 2026DynamicMem: A Long-Horizon Memory Benchmark in Real-World Settingscited by 4
  17. 2026A-TMA: Decoupling State-Aware Memory Failures in Long-Term Agent Memorycited by 4
  18. 2025The Narrative Continuity Test: A Conceptual Framework for Evaluating Identity Persistence in AI Systemscited by 3
  19. 2026GLOVE: Global Verifier for LLM Memory-Environment Realignmentcited by 3
  20. 2026Mem-W: Latent Memory-Native GUI Agentscited by 3
  21. 2026Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agentscited by 3
  22. 2026AgentOdyssey: Open-Ended Long-Horizon Text Game Generation for Test-Time Continual Learning Agentscited by 3
  23. 2026Simulating Human Memory with Language Modelscited by 2
  24. 2026REAL: A Reasoning-Enhanced Graph Framework for Long-Term Memory Management of LLMscited by 2
  25. 2026EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memorycited by 2
  26. 2026Retrieval-Driven Memory Reconsolidation for Long-Term LLM Agentscited by 2
  27. 2025Narrative Memory in Machines: Multi-Agent Arc Extraction in Serialized TVcited by 1
  28. 2026Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memorycited by 1
  29. 2026LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inferencecited by 1
  30. 2026RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agentscited by 1
  31. 2026ME-IQA: Memory-Enhanced Image Quality Assessment via Re-Rankingcited by 0
  32. 2026Connecting the Dots: Benchmarking Reflective Memory in Long-Horizon Dialoguecited by 0
  33. 2026Joint Agent Memory and Exploration Learning via Novelty Signalscited by 0
  34. 2026Token-Operations-Oriented Inference Optimization Techniques for Large Modelscited by 0
  35. 2026Statistical Adversaries: Natural Backdoor-like Adversarial Features in Clean Vision Datasetscited by 0
  36. 2026TokenMem: Faithful Knowledge Injection for Frozen LLMscited by 0
  37. 2026LifeFuse-Mem: Lifecycle-Aware State Fusion Against Temporary Overwriting for Long-Term Memorycited by 0
  38. 2026SpeakerMem-R1: Speaker-Centered Dual-Track Memory for Multi-Party Dialoguecited by 0

50 most cited of 54 All 54 (JSON)