# Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning

arXiv: https://arxiv.org/abs/2605.06130 · HTML: https://mlsanity.com/paper/2605.06130/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 57 of its 65 references that are in the map.
- First posted on arXiv 7 May 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Agent skill learning and evolution](https://mlsanity.com/agents/agent-skill-learning-and-evolution/)

## Builds on (most cited first, 30 of 57)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [Group-in-Group Policy Optimization for LLM Agent Training](https://mlsanity.com/paper/2505.10978/) (2025)
- [RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2504.20073/) (2025)
- [Reinforcement Learning via Self-Distillation](https://mlsanity.com/paper/2601.20802/) (2026)
- [SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning](https://mlsanity.com/paper/2602.08234/) (2026)
- [ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL](https://mlsanity.com/paper/2402.19446/) (2024)
- [Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents](https://mlsanity.com/paper/2408.07199/) (2024)
- [Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents](https://mlsanity.com/paper/2403.02502/) (2024)
- [SimpleMem: Efficient Lifelong Memory for LLM Agents](https://mlsanity.com/paper/2601.02553/) (2026)
- [MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents](https://mlsanity.com/paper/2602.02474/) (2026)
- [EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle](https://mlsanity.com/paper/2510.16079/) (2025)
- [Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory](https://mlsanity.com/paper/2511.20857/) (2025)
- [SoK: Agentic Skills -- Beyond Tool Use in LLM Agents](https://mlsanity.com/paper/2602.20867/) (2026)
- [Memento: Fine-tuning LLM Agents without Fine-tuning LLMs](https://mlsanity.com/paper/2508.16153/) (2025)
- [AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution](https://mlsanity.com/paper/2603.01145/) (2026)
- [Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward](https://mlsanity.com/paper/2602.12430/) (2026)

## Built on by (most cited first, 30 of 35)

- [Self-Distilled Agentic Reinforcement Learning](https://mlsanity.com/paper/2605.15155/) (2026)
- [MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation](https://mlsanity.com/paper/2605.27366/) (2026)
- [SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution](https://mlsanity.com/paper/2605.18401/) (2026)
- [Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories](https://mlsanity.com/paper/2608.02276/) (2026)
- [UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation](https://mlsanity.com/paper/2606.29502/) (2026)
- [WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution](https://mlsanity.com/paper/2608.27454/) (2026)
- [SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing](https://mlsanity.com/paper/2606.14239/) (2026)
- [Skill0.5: Joint Skill Internalization and Utilization for Out-of-Distribution Generalization in Agentic Reinforcement Learning](https://mlsanity.com/paper/2605.28424/) (2026)
- [SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution](https://mlsanity.com/paper/2607.26784/) (2026)
- [Parametric Skills](https://mlsanity.com/paper/2606.30015/) (2026)
- [SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation](https://mlsanity.com/paper/2608.02287/) (2026)
- [What Is a Skill Worth? Structure-Aware Shapley Valuation of Agent Skills](https://mlsanity.com/paper/2608.04562/) (2026)
- [AHEAD: Adaptive Hindsight with Environment-Augmented Distillation for Agentic RL](https://mlsanity.com/paper/2608.24114/) (2026)
- [Harnessing Agent Skills: Architectural Patterns and a Reference Architecture for Skill-Mediated LLM Agents](https://mlsanity.com/paper/2606.20631/) (2026)
- [Learning Globally Reusable Skills for Coding Agents](https://mlsanity.com/paper/2608.06153/) (2026)
- [SkillAligner: Treating Retrieved Skills as Adaptable Drafts at Execution Time](https://mlsanity.com/paper/2608.06880/) (2026)
- [SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents](https://mlsanity.com/paper/2608.18852/) (2026)
- [PR-OPD: Privileged Representation On-policy Self-Distillation for Agentic Reinforcement Learning](https://mlsanity.com/paper/2609.36642/) (2026)
- [When Is Enough Enough in Self-Evolving LLM Systems?](https://mlsanity.com/paper/2610.04756/) (2026)
- [From Execution to Capability: Scientific Experience Consolidation via Procedural Knowledge Synthesis](https://mlsanity.com/paper/2607.24459/) (2026)
- [CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolution](https://mlsanity.com/paper/2609.04865/) (2026)
- [RLHarness: Co-evolving Procedural Skills with Reinforcement Learning for Long-horizon Multimodal Reasoning](https://mlsanity.com/paper/2609.32326/) (2026)
- [PluginRSI: Recursive Improvement of Agent Harnesses with Reusable Plugins](https://mlsanity.com/paper/2609.32423/) (2026)
- [Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses](https://mlsanity.com/paper/2609.36630/) (2026)
- [SkillCome: Group Contrast Skill Optimization with Dual Memory](https://mlsanity.com/paper/2609.37128/) (2026)
- [From Learner Behavior to Reusable Skills for Effective and Efficient Learner Simulation](https://mlsanity.com/paper/2609.37157/) (2026)
- [SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation](https://mlsanity.com/paper/2609.37539/) (2026)
- [Beyond Instruction Following: Learning Grounded Skill-Following with Skill Contracts](https://mlsanity.com/paper/2610.05161/) (2026)
- [SkillCycle: Co-Evolving Agent Policies and Skill Banks](https://mlsanity.com/paper/2610.09430/) (2026)
- [Learning to Accumulate Knowledge with Mutual Information](https://mlsanity.com/paper/2610.10042/) (2026)

## Abstract (arXiv)

A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in isolation or with separate reward sources, resulting in partial and conflicting evolution. We propose Skill1, a framework that trains a single policy to co-evolve skill selection, utilization, and distillation toward a shared task-outcome objective. The policy generates a query to search the skill library, re-ranks candidates to select one, solves the task conditioned on it, and distills a new skill from the trajectory. All learning derives from a single task-outcome signal. Its low-frequency trend credits selection and its high-frequency variation credits distillation. Experiments on ALFWorld and WebShop show that Skill1 outperforms prior skill-based and reinforcement learning baselines. Training dynamics confirm the co-evolution of the three capabilities, and ablations show that removing any credit signal degrades the evolution.