mlsanityData to 7 Oct 2026

Human-inspired Perspectives: A Survey on AI Long-term Memory

arXiv 2411.00489 ↗First posted 1 Nov 2024cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

With the rapid advancement of AI systems, their abilities to store, retrieve, and utilize information over the long term - referred to as long-term memory - have become increasingly significant. These capabilities are crucial for enhancing the performance of AI systems across a wide range of tasks. However, there is currently no comprehensive survey that systematically investigates AI's long-term memory capabilities, formulates a theoretical framework, and inspires the development of next-generation AI long-term memory systems. This paper begins by introducing the mechanisms of human long-term memory, then explores AI long-term memory mechanisms, establishing a mapping between the two. Based on the mapping relationships identified, we extend the current cognitive architectures and propose the Cognitive Architecture of Self-Adaptive Long-term Memory (SALM). SALM provides a theoretical framework for the practice of AI long-term memory and holds potential for guiding the creation of next-generation long-term memory driven AI systems. Finally, we delve into the future directions and application prospects of AI long-term memory.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 162 of its 347 references that are in the map. First posted on arXiv 1 Nov 2024.

Builds on

162 of 347 references.

References · 162

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  10. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  11. 2013Auto-Encoding Variational Bayescited by 3,980
  12. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
38 more
  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2017Billion-scale similarity search with GPUscited by 2,477
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2019Language Models as Knowledge Bases?cited by 1,922
  10. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  12. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  13. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  14. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  15. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  16. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  17. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  18. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  19. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  20. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  21. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  22. 2016An overview of gradient descent optimization algorithmscited by 1,021
  23. 2019Machine Unlearningcited by 1,020
  24. 2021Scaling Vision Transformerscited by 993
  25. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  26. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  27. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  28. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  29. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  30. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  31. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  32. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  33. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  34. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  35. 2014Neural Turing Machinescited by 630
  36. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  37. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  38. 2016Self-critical Sequence Training for Image Captioningcited by 544

50 most cited of 162 All 162 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 6.

12 papers.

  1. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  2. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
  3. 2025Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topicscited by 36
  4. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  5. 2026HiMem: Hierarchical Long-Term Memory for LLM Long-Horizon Agentscited by 16
  6. 2025Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architecturescited by 9
  7. 2025Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impactcited by 6
  8. 2025Smarter Together: Creating Agentic Communities of Practice through Shared Experiential Learningcited by 5
  9. 2026Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedbackcited by 1
  10. 2025Wormhole Memory: A Rubik's Cube for Cross-Dialogue Retrievalcited by 0
  11. 2026Expanding Spatial and Temporal Context for Robotic Imitation Learning With Scene Graphscited by 0
  12. 2026Agentic Context Management: Solving Agent Memory and Cost by Treating Them as Lifecycle and Architecture Problemscited by 0