mlsanityData to 14 Oct 2026

Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

arXiv 2511.20857 ↗First posted 25 Nov 2025cs.CL cs.AIcited by 136

On the map: LLM agents and tool use → Agent memory systems

also in 1 more
Abstract (arXiv)

Statefulness is essential for large language model (LLM) agents to perform long-term planning and problem-solving. This makes memory a critical component, yet its management and evolution remain largely underexplored. Existing evaluations mostly focus on static conversational settings, where memory is passively retrieved from dialogue to answer queries, overlooking the dynamic ability to accumulate and reuse experience across evolving task streams. In real-world environments such as interactive problem assistants or embodied agents, LLMs are required to handle continuous task streams, yet often fail to learn from accumulated interactions, losing valuable contextual insights, a limitation that calls for test-time evolution, where LLMs retrieve, integrate, and update memory continuously during deployment. To bridge this gap, we introduce Evo-Memory, a comprehensive streaming benchmark and framework for evaluating self-evolving memory in LLM agents. Evo-Memory structures datasets into sequential task streams, requiring LLMs to search, adapt, and evolve memory after each interaction. We unify and implement over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets. To better benchmark experience reuse, we provide a baseline method, ExpRAG, for retrieving and utilizing prior experience, and further propose ReMem, an action-think-memory refine pipeline that tightly integrates reasoning, task actions, and memory updates to achieve continual improvement.

Key facts. Cited by 136 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 40 of its 52 references that are in the map. First posted on arXiv 25 Nov 2025.

Builds on

40 of 52 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning27 Aug
  2. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions7 Jul
  3. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents18 Jun
  4. 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners17 May
  5. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory28 Apr
  6. 2025A-MEM: Agentic Memory for LLM Agents17 Feb
  7. 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memory20 Jan
  8. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb

Other references · 32

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  10. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  11. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  12. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
20 more
  1. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  2. 2023Large Language Modelscited by 780
  3. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  4. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  5. 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
  6. 2024Agent Workflow Memorycited by 383
  7. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 377
  8. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  9. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
  10. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 244
  11. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  12. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  13. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 163
  14. 2023SCM: Enhancing Large Language Model with Self-Controlled Memory Frameworkcited by 98
  15. 2024RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graphcited by 91
  16. 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Modelscited by 73
  17. 2023RET-LLM: Towards a General Read-Write Memory for Large Language Modelscited by 64
  18. 2024StreamBench: Towards Benchmarking Continuous Improvement of Language Agentscited by 47
  19. 2025EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systemscited by 37
  20. 2025Enabling Self-Improving Agents to Learn at Test Time With Human-In-The-Loop Guidancecited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 133.

136 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
  3. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 142
  4. 2026MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memorycited by 129
  5. 2026MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Taskscited by 84
  6. 2026Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agentscited by 54
  7. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  8. 2026Meta Context Engineering via Agentic Skill Evolutioncited by 49
  9. 2026SkillOS: Learning Skill Curation for Self-Evolving Agentscited by 49
  10. 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injectionscited by 48
  11. 2026Code as Agent Harnesscited by 47
  12. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 44
38 more
  1. 2026Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learningcited by 35
  2. 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agentscited by 34
  3. 2026Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineeringcited by 28
  4. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 25
  5. 2026SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agentscited by 25
  6. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 21
  7. 2026RealMem: Benchmarking LLMs in Real-World Memory-Driven Interactioncited by 20
  8. 2025MemR3: Memory Retrieval via Reflective Reasoning for LLM Agentscited by 19
  9. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 18
  10. 2026EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspectivecited by 17
  11. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  12. 2026Position: Agentic Evolution is the Path to Evolving LLMscited by 16
  13. 2026Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agentscited by 16
  14. 2026EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agentscited by 15
  15. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  16. 2026Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RLcited by 12
  17. 2026Evaluating Memory Structure in LLM Agentscited by 12
  18. 2026Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Tracescited by 12
  19. 2026M^star: Every Task Deserves Its Own Memory Harnesscited by 11
  20. 2026EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfercited by 11
  21. 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvementcited by 11
  22. 2026TAME: A Trustworthy Test-Time Evolution of Agent Memory with Systematic Benchmarkingcited by 10
  23. 2026Auto-Dreamer: Learning Offline Memory Consolidation for Language Agentscited by 10
  24. 2026From Procedural Skills to Strategy Genes: Towards Experience-Driven Test-Time Evolutioncited by 9
  25. 2026AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agentscited by 9
  26. 2026PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agentscited by 9
  27. 2026EvoClinician: A Self-Evolving Agent for Multi-Turn Medical Diagnosis via Test-Time Evolutionary Learningcited by 8
  28. 2026Live-Evo: Online Evolution of Agentic Memory from Continuous Feedbackcited by 8
  29. 2026Risky-Bench: Probing Agentic Safety Risks under Real-World Deploymentcited by 8
  30. 2026UMEM: Unified Memory Extraction and Management Framework for Generalizable Memorycited by 8
  31. 2026ARC: Active and Reflection-driven Context Management for Long-Horizon Information Seeking Agentscited by 7
  32. 2026Retrieval-Augmented LLM Agents: Learning to Learn from Experiencecited by 7
  33. 2026Memory Intelligence Agentcited by 7
  34. 2026Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloadscited by 7
  35. 2026Evo-MedAgent: Beyond One-Shot Diagnosis with Agents That Remember, Reflect, and Improvecited by 6
  36. 2026Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptationcited by 6
  37. 2026Aligning Agentic World Models via Knowledgeable Experience Learningcited by 5
  38. 2026Can Large Language Models Keep Up? Benchmarking Online Adaptation to Continual Knowledge Streamscited by 5

50 most cited of 136 All 136 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Memory in the LLM Era: Modular Architectures and Strategies in a Unified Framework
  2. 2026EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments
  3. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
  4. 2025Rethinking Memory in LLM based Agents: Representations, Operations, and Emerging Topics
  5. 2026Can Agent Memory Systems Track Evolving State?
  6. 2026MemGym: a Long-Horizon Memory Environment for LLM Agents
  7. 2026AdaMEM: Test-Time Adaptive Memory for Language Agents
  8. 2026ERSkill: Evolving for Skill-Guided Adaptive Memory Retrieval