mlsanityData to 7 Oct 2026

SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

arXiv 2602.08234 ↗First posted 9 Feb 2026cs.LGcited by 252

On the map: LLM agents and tool use → Agent skill learning and evolution

Abstract (arXiv)

Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents agents from extracting high-level, reusable behavioral patterns that are essential for generalization. In this paper, we propose SkillRL, a framework that bridges the gap between raw experience and policy improvement through automatic skill discovery and recursive evolution. Our approach introduces an experience-based distillation mechanism to build a hierarchical skill library SkillBank, an adaptive retrieval strategy for general and task-specific heuristics, and a recursive evolution mechanism that allows the skill library to co-evolve with the agent's policy during reinforcement learning. These innovations significantly reduce the token footprint while enhancing reasoning utility. Experimental results on ALFWorld, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance, outperforming strong baselines over 15.3% and maintaining robustness as task complexity increases. Code is available at this https://github.com/aiming-lab/SkillRL.

Key facts. Cited by 252 papers in the map (data through 7 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 39 of its 51 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

39 of 51 references.

References · 39

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2023Qwen Technical Reportcited by 3,273
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  9. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  11. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  12. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
27 more
  1. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  2. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  3. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  6. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  7. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  8. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  9. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  10. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  11. 2024Agent Workflow Memorycited by 342
  12. 2025Memory in the Age of AI Agentscited by 285
  13. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  14. 2025ZeroSearch: Incentivize the Search Capability of LLMs without Searchingcited by 203
  15. 2025MIRIX: Multi-Agent Memory System for LLM-Based Agentscited by 160
  16. 2026SimpleMem: Efficient Lifelong Memory for LLM Agentscited by 140
  17. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 129
  18. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  19. 2026MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memorycited by 113
  20. 2025G-Memory: Tracing Hierarchical Memory for Multi-Agent Systemscited by 112
  21. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 90
  22. 2025Memp: Exploring Agent Procedural Memorycited by 88
  23. 2025StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimizationcited by 82
  24. 2025MemEvolve: Meta-Evolution of Agent Memory Systemscited by 81
  25. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 68
  26. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  27. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 252.

252 papers.

Sort:

By area:

  1. 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skillscited by 161
  2. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 122
  3. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnessescited by 119
  4. 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verificationcited by 117
  5. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 82
  6. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 76
  7. 2026Natural-Language Agent Harnessescited by 54
  8. 2026Self-Distilled Agentic Reinforcement Learningcited by 49
  9. 2026SkillNet: Create, Evaluate, and Connect AI Skillscited by 48
  10. 2026SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Taskscited by 45
  11. 2026SkillOS: Learning Skill Curation for Self-Evolving Agentscited by 45
  12. 2026MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wildcited by 35
38 more
  1. 2026Dynamic Dual-Granularity Skill Bank for Agentic RLcited by 33
  2. 2026SkillGen: Verified Inference-Time Agent Skill Synthesiscited by 29
  3. 2026ARISE: Agent Reasoning with Intrinsic Skill Evolution in Hierarchical Reinforcement Learningcited by 28
  4. 2026Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learningcited by 28
  5. 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agentscited by 28
  6. 2026GraSP: Graph-Structured Skill Compositions for LLM Agentscited by 25
  7. 2026AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaborationcited by 23
  8. 2026SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revisioncited by 21
  9. 2026SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agentscited by 19
  10. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 19
  11. 2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolutioncited by 18
  12. 2026From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skillscited by 18
  13. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 17
  14. 2026SkillGenBench: Benchmarking Skill Generation Pipelines for LLM Agentscited by 17
  15. 2026RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedbackcited by 16
  16. 2026GEMS: Agent-Native Multimodal Generation with Memory and Skillscited by 16
  17. 2026Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agentscited by 16
  18. 2026SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphscited by 15
  19. 2026Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Taskscited by 13
  20. 2026MMSkills: Towards Multimodal Skills for General Visual Agentscited by 13
  21. 2026Terminal-World: Scaling Terminal-Agent Environments via Agent Skillscited by 13
  22. 2026ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RLcited by 13
  23. 2026Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectoriescited by 13
  24. 2026Skill-R1: Agent Skill Evolution via Reinforcement Learningcited by 12
  25. 2026SkillEvolver: Skill Learning as a Meta-Skillcited by 12
  26. 2026MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolutioncited by 12
  27. 2026EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agentscited by 11
  28. 2026SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditingcited by 11
  29. 2026ClawArena: Benchmarking AI Agents in Evolving Information Environmentscited by 10
  30. 2026Dynamic Skill Lifecycle Management for Agentic Reinforcement Learningcited by 10
  31. 2026UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillationcited by 10
  32. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environmentscited by 9
  33. 2026OpenSkill: Open-World Self-Evolution for LLM Agentscited by 9
  34. 2026PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoningcited by 9
  35. 2026Combee: Scaling Prompt Learning for Self-Improving Language Model Agentscited by 8
  36. 2026SkillGraph: Self-Evolving Multi-Agent Collaboration with Multimodal Graph Topologycited by 8
  37. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 8
  38. 2026SkillMaster: Toward Autonomous Skill Mastery in LLM Agentscited by 8

50 most cited of 252 All 252 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?
  2. 2025Reinforcement Learning for Self-Improving Agent with Skill Library
  3. 2026SkillEvoReg: Regularizing Agent Skill Evolution Against Overfitting
  4. 2026SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion
  5. 2026SkillX: Automatically Constructing Skill Knowledge Bases for Agents
  6. 2026Skill0.5: Joint Skill Internalization and Utilization for Out-of-Distribution Generalization in Agentic Reinforcement Learning
  7. 2025Automated Skill Discovery for Language Agents through Exploration and Iterative Feedback
  8. 2026SkillFocus: Evolving Agent Skills via Capability Decomposition