mlsanityData to 14 Oct 2026

LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners

arXiv 2505.11942 ↗First posted 17 May 2025cs.AIcited by 78

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Lifelong learning is essential for intelligent agents operating in dynamic environments. Current large language model (LLM)-based agents, however, remain stateless and unable to accumulate or transfer knowledge over time. Existing benchmarks treat agents as static systems and fail to evaluate lifelong learning capabilities. We present LifelongAgentBench, the first unified benchmark designed to systematically assess the lifelong learning ability of LLM agents. It provides skill-grounded, interdependent tasks across three interactive environments, Database, Operating System, and Knowledge Graph, with automatic label verification, reproducibility, and modular extensibility. Extensive experiments reveal that conventional experience replay has limited effectiveness for LLM agents due to irrelevant information and context length constraints. We further introduce a group self-consistency mechanism that significantly improves lifelong learning performance. We hope LifelongAgentBench will advance the development of adaptive, memory-capable LLM agents.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 18 of its 22 references that are in the map. First posted on arXiv 17 May 2025.

Builds on

18 of 22 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Lifelong Learning of Large Language Model based Agents: A Roadmap13 Jan

Other references · 17

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  8. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  9. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  10. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  11. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  12. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 171
5 more
  1. 2024Tree Search for Language Model Agentscited by 151
  2. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 114
  3. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agentscited by 103
  4. 2024DebugBench: Evaluating Debugging Capability of Large Language Modelscited by 76
  5. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 64

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 68.

78 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  3. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 136
  4. 2026MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memorycited by 129
  5. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  6. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 44
  7. 2026MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluationcited by 27
  8. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 25
  9. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 21
  10. 2025Context Engineering 2.0: The Context of Context Engineeringcited by 18
  11. 2026Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environmentscited by 17
  12. 2026SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessmentcited by 14
38 more
  1. 2026GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agentscited by 13
  2. 2025Continual Learning for Generative AI: From LLMs to MLLMs and Beyondcited by 12
  3. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  4. 2026SEAGym: An Evaluation Environment for Self-Evolving LLM Agentscited by 11
  5. 2026EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfercited by 11
  6. 2026GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)cited by 10
  7. 2026All-Mem: Agentic Lifelong Memory via Dynamic Topology Evolutioncited by 9
  8. 2026LLMs Corrupt Your Documents When You Delegatecited by 9
  9. 2026AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agentscited by 9
  10. 2026PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agentscited by 9
  11. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  12. 2026RetailBench: Benchmarking long horizon reasoning and coherent decision making of LLM agents in realistic retail environmentscited by 6
  13. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
  14. 2026Self-Consolidation for Self-Evolving Agentscited by 5
  15. 2026Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streamscited by 5
  16. 2026SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent Systemcited by 5
  17. 2026MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systemscited by 5
  18. 2026Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgentscited by 5
  19. 2026SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolutioncited by 5
  20. 2026Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environmentcited by 4
  21. 2026Scaling Teams or Scaling Time? Memory Enabled Lifelong Learning in LLM Multi-Agent Systemscited by 4
  22. 2026Learning While Acting: A Skill-Enhanced Test-Time Co-Evolution Framework for Online Lifelong Learning Agentscited by 4
  23. 2026CoEvo-Mem: Co-Evolving Retrieval Policy and Memory Bank for LLM Agentscited by 4
  24. 2026Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Developmentcited by 4
  25. 2026ARM: Role-Conditioned Neuron Transplantation for Training-Free Generalist LLM Agent Mergingcited by 3
  26. 2026Mem-π: Adaptive Memory through Learning When and What to Generatecited by 3
  27. 2026FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewardscited by 3
  28. 2026PATH-Bench: Path-Dependent Evaluation of Lifelong Agentscited by 3
  29. 2026FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflowscited by 3
  30. 2026VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?cited by 3
  31. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  32. 2026From Agent Loops to Deterministic Graphs: Execution Lineage for Reproducible AI-Native Workcited by 2
  33. 2026M3Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Taskscited by 2
  34. 2026From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memorycited by 2
  35. 2026LatentGym: A Testbed For Cross-Task Experiential Learning With Controllable Latent Structurecited by 2
  36. 2026RoMeRL: Balancing Feedback Coverage and the Memory-Reward Trap in Self-Evolving Agent Memory via Reduced-Order Utility Statescited by 2
  37. 2026PACE-Bench: Benchmarking Physics Adaptation via Code Evolution in Dynamic Environmentscited by 2
  38. 2025CurLL: A Developmental Framework to Evaluate Continual Learning in Language Modelscited by 1

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023AgentBench: Evaluating LLMs as Agents
  2. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
  3. 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution
  4. 2025Enabling Personalized Long-term Interactions in LLM-based Agents through Persistent Memory and User Profiles
  5. 2025Intrinsic Memory Agents: Heterogeneous Multi-Agent LLM Systems through Structured Contextual Memory
  6. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
  7. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
  8. 2023A Survey on Large Language Model based Autonomous Agents