mlsanityData to 7 Oct 2026

EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle

arXiv 2510.16079 ↗First posted 17 Oct 2025cs.CL cs.AIcited by 129

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

Current Large Language Model (LLM) agents show strong performance in tool use, but lack the crucial capability to systematically learn from their own experiences. While existing frameworks mainly focus on mitigating external knowledge gaps, they fail to address a more fundamental limitation: the inability to iteratively refine problem-solving strategies. In this work, we introduce EvolveR, a framework designed to enable agent to self-improve through a complete, closed-loop experience lifecycle. This lifecycle comprises two key stages: (1) Offline Self-Distillation, where the agent's interaction trajectories are synthesized into a structured repository of abstract, reusable strategic principles; (2) Online Interaction, where the agent interacts with tasks and actively retrieves distilled principles to guide its decision-making, accumulating a diverse set of behavioral trajectories. This loop employs a policy reinforcement mechanism to iteratively update the agent based on its performance. We demonstrate the effectiveness of EvolveR on complex multi-hop question-answering benchmarks, where it achieves superior performance over strong agentic baselines. Our work presents a comprehensive blueprint for agents that learn not only from external data but also from the consequences of their own actions, paving the way for more autonomous and continuously improving systems. Code is available at https://github.com/Edaizi/EvolveR.

Key facts. Cited by 129 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 30 of its 41 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

30 of 41 references.

References · 30

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  11. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  12. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
18 more
  1. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  2. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  4. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  5. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  6. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  7. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  8. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  9. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  10. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  11. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  12. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learningcited by 220
  13. 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systemscited by 112
  14. 2025CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answeringcited by 36
  15. 2023Introspective Tips: Large Language Model for In-Context Decision Makingcited by 28
  16. 2025O2-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answeringcited by 22
  17. 2024Boosting Large Language Models with Continual Learning for Aspect-based Sentiment Analysiscited by 10
  18. 2025Task-Core Memory Management and Consolidation for Long-term Continual Learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 124.

129 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  3. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 129
  4. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 122
  5. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 82
  6. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 81
  7. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 76
  8. 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agentscited by 75
  9. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 71
  10. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 68
  11. 2026Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agentscited by 44
  12. 2026MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wildcited by 35
38 more
  1. 2026Dynamic Dual-Granularity Skill Bank for Agentic RLcited by 33
  2. 2026ARISE: Agent Reasoning with Intrinsic Skill Evolution in Hierarchical Reinforcement Learningcited by 28
  3. 2026Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learningcited by 28
  4. 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agentscited by 28
  5. 2026AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaborationcited by 23
  6. 2026SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revisioncited by 21
  7. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 19
  8. 2026From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skillscited by 18
  9. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 17
  10. 2026RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedbackcited by 16
  11. 2026Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agentscited by 16
  12. 2026SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphscited by 15
  13. 2026ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RLcited by 13
  14. 2026Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectoriescited by 13
  15. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  16. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 12
  17. 2025Agentic Learner with Grow-and-Refine Multimodal Semantic Memorycited by 11
  18. 2026EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agentscited by 11
  19. 2026Evolving Medical Imaging Agents via Experience-driven Self-skill Discoverycited by 10
  20. 2026GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)cited by 10
  21. 2026Dynamic Skill Lifecycle Management for Agentic Reinforcement Learningcited by 10
  22. 2026Library Drift: Diagnosing and Fixing a Silent Failure Mode in Self-Evolving LLM Skill Librariescited by 10
  23. 2026OEP: Poisoning Self-Evolving LLM Agents via Locally Correct but Non-Transferable Experiencescited by 9
  24. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 9
  25. 2026Position: Beyond Model-Centric Prediction -- Agentic Time Series Forecastingcited by 8
  26. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 8
  27. 2026SkillMaster: Toward Autonomous Skill Mastery in LLM Agentscited by 8
  28. 2026UMEM: Unified Memory Extraction and Management Framework for Generalizable Memorycited by 7
  29. 2026Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agentscited by 7
  30. 2026LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agentscited by 7
  31. 2026Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remembercited by 7
  32. 2026Principle-Evolvable Scientific Discovery via Uncertainty Minimizationcited by 6
  33. 2026The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societiescited by 6
  34. 2026SAGE: Multi-Agent Self-Evolution for LLM Reasoningcited by 6
  35. 2026CODESKILL: Learning Self-Evolving Skills for Coding Agentscited by 6
  36. 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studiescited by 6
  37. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
  38. 2026SPIRAL: Self-Evolving Action-Conditioned Video Generation via Reflective Planning Agentscited by 5

50 most cited of 129 All 129 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Consolidation for Self-Evolving Agents
  2. 2024A Survey on Self-Evolution of Large Language Models
  3. 2024Benchmark Self-Evolving: A Multi-Agent Framework for Dynamic LLM Evaluation
  4. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolution
  5. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
  6. 2026Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordination
  7. 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent
  8. 2026MetaEvo: A Meta-Optimization Framework for Experience-Driven Agent Evolution