mlsanityData to 7 Oct 2026

Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills

arXiv 2402.03244 ↗First posted 5 Feb 2024cs.LG cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have recently been used for sequential decision making in interactive environments. However, leveraging environment reward signals for continual LLM actor improvement is not straightforward. We propose Skill Set Optimization (SSO) for improving LLM actor performance through constructing and refining sets of transferable skills. SSO constructs skills by extracting common subtrajectories with high rewards and generating subgoals and instructions to represent each skill. These skills are provided to the LLM actor in-context to reinforce behaviors with high rewards. Then, SSO further refines the skill set by pruning skills that do not continue to result in high rewards. We evaluate our method in the classic videogame NetHack and the text environment ScienceWorld to demonstrate SSO's ability to optimize a set of skills and perform in-context policy improvement. SSO outperforms baselines by 40% in our custom NetHack task and outperforms the previous state-of-the-art in ScienceWorld by 35%.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 5 Feb 2024.

Builds on

19 of 26 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  6. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  7. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  8. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  9. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  10. 2023Language Models can Solve Computer Taskscited by 371
  11. 2022CodeT: Code Generation with Generated Testscited by 369
  12. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
7 more
  1. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  2. 2020The NetHack Learning Environmentcited by 196
  3. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  4. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 172
  5. 2021MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Researchcited by 96
  6. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  7. 2023CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalizationcited by 88

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 12.

24 papers.

Sort:

By area:

  1. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  2. 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skillscited by 161
  3. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 76
  4. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 74
  5. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
  6. 2024VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thoughtcited by 55
  7. 2024SelfGoal: Your Language Agents Already Know How to Achieve High-level Goalscited by 21
  8. 2025The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvementcited by 19
  9. 2024Strategist: Self-improvement of LLM Decision Making via Bi-Level Tree Searchcited by 16
  10. 2024AgentKit: Structured LLM Reasoning with Dynamic Graphscited by 12
  11. 2024Enhancing Agent Learning through World Dynamics Modelingcited by 9
  12. 2025Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learningcited by 9
12 more
  1. 2025H2R: Hierarchical Hindsight Reflection for Multi-Task LLM Agentscited by 8
  2. 2026Skill-to-LoRA: From Using Skills to Learning Behaviors for Token-Efficient LLM Agentscited by 8
  3. 2026When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agentscited by 7
  4. 2026ABSTRAL: Automatic Design of Multi-Agent Systems Through Iterative Refinement and Topology Optimizationcited by 6
  5. 2024MindForge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learningcited by 5
  6. 2026SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Groundingcited by 4
  7. 2026Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agentscited by 1
  8. 2025CDT: A Comprehensive Capability Framework for Large Language Models Across Cognition, Domain, and Taskcited by 0
  9. 2026Towards Scalable Customization and Deployment of Multi-Agent Systems for Enterprise Applicationscited by 0
  10. 2026SkillShapley: Boundary-Adaptive Shapley Valuation for Skill Step Attribution in LLM Agentscited by 0
  11. 2026TRUSS: Towards Task-Reliable and User-Safe Automated Agent Skill Generationcited by 0
  12. 2026CoSkill: Joint Reinforcement Learning of Reasoning and Meta-Skill Agents for Hierarchical Skill Evolutioncited by 0