mlsanityData to 7 Oct 2026

Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning

arXiv 2605.06130 ↗First posted 7 May 2026cs.AIcited by 28

On the map: LLM agents and tool use → Agent skill learning and evolution

Abstract (arXiv)

A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in isolation or with separate reward sources, resulting in partial and conflicting evolution. We propose Skill1, a framework that trains a single policy to co-evolve skill selection, utilization, and distillation toward a shared task-outcome objective. The policy generates a query to search the skill library, re-ranks candidates to select one, solves the task conditioned on it, and distills a new skill from the trajectory. All learning derives from a single task-outcome signal. Its low-frequency trend credits selection and its high-frequency variation credits distillation. Experiments on ALFWorld and WebShop show that Skill1 outperforms prior skill-based and reinforcement learning baselines. Training dynamics confirm the co-evolution of the three capabilities, and ablations show that removing any credit signal degrades the evolution.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 57 of its 65 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

57 of 65 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization2 Apr
  2. 2026ARISE: Agent Reasoning with Intrinsic Skill Evolution in Hierarchical Reinforcement Learning17 Mar
  3. 2026Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale2 Mar
  4. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agents24 Feb
  5. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward12 Feb
  6. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning9 Feb
  7. 2025Reinforcement Learning for Self-Improving Agent with Skill Library18 Dec

Other references · 50

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  10. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  11. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  12. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
38 more
  1. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  2. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  3. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  4. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 442
  5. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  6. 2026Reinforcement Learning via Self-Distillationcited by 285
  7. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  8. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  9. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  10. 2026SimpleMem: Efficient Lifelong Memory for LLM Agentscited by 140
  11. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 129
  12. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  13. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 129
  14. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
  15. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 110
  16. 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolutioncited by 105
  17. 2025Reinforcement Learning for Long-Horizon Interactive LLM Agentscited by 103
  18. 2023A Definition of Continual Reinforcement Learningcited by 98
  19. 2025Memp: Exploring Agent Procedural Memorycited by 88
  20. 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedbackcited by 86
  21. 2022Partially Observable Markov Decision Processes in Robotics: A Surveycited by 81
  22. 2025Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignmentcited by 77
  23. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 71
  24. 2025SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attributioncited by 68
  25. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Frameworkcited by 58
  26. 2025ExGRPO: Learning to Reason from Experiencecited by 49
  27. 2025Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agentscited by 43
  28. 2025Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Explorationcited by 33
  29. 2026LongCat-Flash-Thinking-2601 Technical Reportcited by 32
  30. 2026Online Experiential Learning for Language Modelscited by 24
  31. 2026SkillOrchestra: Learning to Route Agents via Skill Transfercited by 22
  32. 2025Meta-RL Induces Exploration in Language Agentscited by 17
  33. 2026Exploratory Memory-Augmented LLM Agent via Hybrid On- and Off-Policy Optimizationcited by 17
  34. 2026RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedbackcited by 16
  35. 2025Truly Self-Improving Agents Require Intrinsic Metacognitive Learningcited by 13
  36. 2025Cogito, Ergo Ludo: An Agent that Learns to Play by Reasoning and Planningcited by 12
  37. 2026Memory Intelligence Agentcited by 6
  38. 2026Complementary RL: Towards Efficient Experience-Driven Agent Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 28.

28 papers.

Sort:

By area:

  1. 2026Self-Distilled Agentic Reinforcement Learningcited by 49
  2. 2026MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluationcited by 23
  3. 2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolutioncited by 18
  4. 2026Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectoriescited by 13
  5. 2026SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditingcited by 11
  6. 2026UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillationcited by 10
  7. 2026WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolutioncited by 7
  8. 2026Skill0.5: Joint Skill Internalization and Utilization for Out-of-Distribution Generalization in Agentic Reinforcement Learningcited by 5
  9. 2026SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolutioncited by 3
  10. 2026AHEAD: Adaptive Hindsight with Environment-Augmented Distillation for Agentic RLcited by 2
  11. 2026Harnessing Agent Skills: Architectural Patterns and a Reference Architecture for Skill-Mediated LLM Agentscited by 1
  12. 2026Parametric Skillscited by 1
16 more
  1. 2026Learning Globally Reusable Skills for Coding Agentscited by 1
  2. 2026SkillAligner: Treating Retrieved Skills as Adaptable Drafts at Execution Timecited by 1
  3. 2026SkillGate: Training In-Policy Skill Selection in Long-Horizon Agentscited by 1
  4. 2026From Execution to Capability: Scientific Experience Consolidation via Procedural Knowledge Synthesiscited by 0
  5. 2026CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolutioncited by 0
  6. 2026RLHarness: Co-evolving Procedural Skills with Reinforcement Learning for Long-horizon Multimodal Reasoningcited by 0
  7. 2026PluginRSI: Recursive Improvement of Agent Harnesses with Reusable Pluginscited by 0
  8. 2026Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnessescited by 0
  9. 2026PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learningcited by 0
  10. 2026SkillCome: Group Contrast Skill Optimization with Dual Memorycited by 0
  11. 2026From Learner Behavior to Reusable Skills for Effective and Efficient Learner Simulationcited by 0
  12. 2026SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generationcited by 0
  13. 2026UniSkill: Learning Actor-Aligned Skill Proposals for an Evolving Policycited by 0
  14. 2026Skill-V: Verifiable Self-Evolving Skill Library for Interactive Agentscited by 0
  15. 2026Which Skill to Distill? SGUID: Selecting a Compact Skill Bank for Model-Skill Co-Evolutioncited by 0
  16. 2026ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skillscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Skill-R1: Agent Skill Evolution via Reinforcement Learning
  2. 2026SkillForge: Co-Evolving Skills and Agents via Dynamic Skill Lifecycles
  3. 2026ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?
  4. 2026SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs
  5. 2026SkillEvoReg: Regularizing Agent Skill Evolution Against Overfitting
  6. 2026SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization
  7. 2026SkillFocus: Evolving Agent Skills via Capability Decomposition
  8. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolver