mlsanityData to 14 Oct 2026

Skill Reuse as Compression in Agentic RL

arXiv 2605.31509 ↗First posted 29 May 2026cs.LG cs.AIcited by 1

On the map: LLM agents and tool use → Agent skill learning and evolution

Abstract (arXiv)

Large language model agents trained with reinforcement learning (RL) often learn brittle, task-specific shortcuts. We hypothesize that agents generalize better when their successful trajectories are structurally compressible, decomposed into a small set of reusable abstract patterns. To formalize this, we introduce ReuseRL, which grounds agentic RL in the Minimum Description Length (MDL) principle. ReuseRL extracts a shared skill dictionary from successful trajectories and augments the RL objective with a segmentation cost, explicitly penalizing idiosyncratic behaviors that encode poorly. We prove a PAC-Bayes bound guaranteeing that a dictionary extracted from successful trajectories has bounded expected description length on future successful behavior. Across ALFWorld, TextWorld-Cooking, and Countdown-Stepwise, ReuseRL improves in- and out-of-distribution success over vanilla GRPO and strong round-length baselines.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 18 of its 37 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

18 of 37 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  7. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  8. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  9. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  10. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  11. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
  12. 2026Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agentscited by 23
6 more
  1. 2024PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Controlcited by 21
  2. 2026RAGEN-2: Reasoning Collapse in Agentic RLcited by 20
  3. 2024Theoretical Analysis of Byte-Pair Encodingcited by 16
  4. 2025LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimizationcited by 11
  5. 2025DYSTIL: Dynamic Strategy Induction with Large Language Models for Reinforcement Learningcited by 7
  6. 2026SkillVLA: Tackling Combinatorial Diversity in Dual-Arm Manipulation via Skill Reusecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Coalition-Aware Skill Reliability for Self-Evolving Agentscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Skill-R1: Agent Skill Evolution via Reinforcement Learning
  2. 2026ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL
  3. 2025Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learning
  4. 2026Learning Agentic Policy from Action Guidance
  5. 2026Cross-Benchmark Transfer from RL on Agentic Coding Tasks
  6. 2026SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution
  7. 2026ECHO: Prune To Act, Trace To Learn With Selective Turn Memory In Agentic RL
  8. 2026SkillZip: Evaluation-Free Skill Compression for Self-Evolving Agents by Discovering Reusable Structure