mlsanityData to 7 Oct 2026

Lifelong Learning of Large Language Model based Agents: A Roadmap

arXiv 2501.07278 ↗First posted 13 Jan 2025cs.AIcited by 78

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Lifelong learning, also known as continual or incremental learning, is a crucial component for advancing Artificial General Intelligence (AGI) by enabling systems to continuously adapt in dynamic environments. While large language models (LLMs) have demonstrated impressive capabilities in natural language processing, existing LLM agents are typically designed for static systems and lack the ability to adapt over time in response to new challenges. This survey is the first to systematically summarize the potential techniques for incorporating lifelong learning into LLM-based agents. We categorize the core components of these agents into three modules: the perception module for multimodal input integration, the memory module for storing and retrieving evolving knowledge, and the action module for grounded interactions with the dynamic environment. We highlight how these pillars collectively enable continuous adaptation, mitigate catastrophic forgetting, and improve long-term performance. This survey provides a roadmap for researchers and practitioners working to develop lifelong learning capabilities in LLM agents, offering insights into emerging trends, evaluation metrics, and application scenarios. Relevant literature and resources are available at hrefthis urlhttps://github.com/qianlima-lab/awesome-lifelong-llm-agent.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 273 of its 395 references that are in the map. First posted on arXiv 13 Jan 2025.

Builds on

273 of 395 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agents21 Apr
  2. 2023Memory Sandbox: Transparent and Interactive Memory Management for Conversational Agents3 Aug
  3. 2023Large Language Models Are Semi-Parametric Reinforcement Learning Agents9 Jun

Other references · 270

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  9. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  10. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  11. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  12. 2020Measuring Massive Multitask Language Understandingcited by 7,768
38 more
  1. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  12. 2016Learning without Forgettingcited by 2,465
  13. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  14. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  15. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  16. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  17. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  18. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  19. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  20. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  21. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  22. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  23. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  24. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  25. 2020Knowledge Distillation: A Surveycited by 1,578
  26. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  27. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  28. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  29. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  30. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  31. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  32. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  33. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  34. 2017Continual Learning with Deep Generative Replaycited by 1,152
  35. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  36. 2018Experience Replay for Continual Learningcited by 1,074
  37. 2015Prioritized Experience Replaycited by 1,070
  38. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029

50 most cited of 270 All 270 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 33; 2026: 45.

78 papers.

Sort:

By area:

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  3. 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolutioncited by 105
  4. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 72
  5. 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 65
  6. 2025Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directionscited by 46
  7. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
  8. 2025MLZero: A Multi-Agent System for End-to-end Machine Learning Automationcited by 22
  9. 2025Agents of Change: Self-Evolving LLM Agents for Strategic Planningcited by 20
  10. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMscited by 18
  11. 2025InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformationcited by 16
  12. 2025A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AIcited by 16
38 more
  1. 2025The Future of Continual Learning in the Era of Foundation Models: Three Key Directionscited by 16
  2. 2025Coarse-to-Fine Grounded Memory for LLM Agent Planningcited by 16
  3. 2025Enhancing Multimodal Continual Instruction Tuning with BranchLoRAcited by 14
  4. 2025GIScience in the Era of Artificial Intelligence: A Research Agenda Towards Autonomous GIScited by 13
  5. 2026PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environmentscited by 13
  6. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  7. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 12
  8. 2026On Safety Risks in Experience-Driven Self-Evolving Agentscited by 12
  9. 2025Analyzing Mitigation Strategies for Catastrophic Forgetting in End-to-End Training of Spoken Language Modelscited by 11
  10. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 11
  11. 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10
  12. 2026E-mem: Multi-agent based Episodic Context Reconstruction for LLM Agent Memorycited by 10
  13. 2025Beyond Profile: From Surface-Level Facts to Deep Persona Simulation in LLMscited by 9
  14. 2025A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomycited by 9
  15. 2025On the Failure of Latent State Persistence in Large Language Modelscited by 8
  16. 2026Live-Evo: Online Evolution of Agentic Memory from Continuous Feedbackcited by 8
  17. 2025Toward Embodied AGI: A Review of Embodied AI and the Road Aheadcited by 7
  18. 2025ID-RAG: Identity Retrieval-Augmented Generation for Long-Horizon Persona Coherence in Generative Agentscited by 7
  19. 2025VisEscape: A Benchmark for Evaluating Exploration-driven Decision-making in Virtual Escape Roomscited by 6
  20. 2026Agent-Dice: Disentangling Knowledge Updates via Geometric Consensus for Agent Continual Learningcited by 6
  21. 2026Retrieval-Augmented LLM Agents: Learning to Learn from Experiencecited by 6
  22. 2026Training Transformers for KV Cache Compressibilitycited by 6
  23. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
  24. 2026Self-Consolidation for Self-Evolving Agentscited by 5
  25. 2026Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streamscited by 5
  26. 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvementcited by 5
  27. 2026Agentic Design Patterns: A System-Theoretic Frameworkcited by 4
  28. 2026Autonomous Continual Learning for Environment Adaptation of Computer-Use Agentscited by 4
  29. 2026Narrow Fine-Tuning Erodes Safety Alignment in Vision-Language Agentscited by 4
  30. 2026Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stagescited by 4
  31. 2026RAG or Learning? Understanding the Limits of LLM Adaptation under Continuous Knowledge Drift in the Real Worldcited by 4
  32. 2026Remembering More, Risking More: Longitudinal Safety Risks in Memory-Equipped LLM Agentscited by 4
  33. 2026Are You Still the Agent I Authorized? Earned Authority under a Fixed Ceiling for Evolving Agentscited by 4
  34. 2025Sophia: A Persistent Agent Framework of Artificial Lifecited by 3
  35. 2026Scaling Teams or Scaling Time? Memory Enabled Lifelong Learning in LLM Multi-Agent Systemscited by 3
  36. 2026PsychAgent: An Experience-Driven Lifelong Learning Agent for Self-Evolving Psychological Counselorcited by 2
  37. 2026MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systemscited by 2
  38. 2026PGMem: Tightly Coupled Persona-Memory Graph for Lifelong Personalized Agentscited by 2

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Incremental Learning in Large Language Models: A Critical Review
  2. 2025Catastrophic Forgetting in LLMs: A Comparative Analysis Across Language Tasks
  3. 2026Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents
  4. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
  5. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
  6. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
  7. 2025Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics
  8. 2026When Does Continual Learning Require Learning