SkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving
Not on the map. Why?
Abstract (arXiv)
Human-written agent skills encode rich workflows for real-world problem solving, but are typically used as external inference-time instructions rather than internalized as reusable model capabilities. We introduce textttSkillGym, a framework that transforms these skills into executable, verifiable training environments for large language model agents. Its skill-to-task pipeline instantiates concrete tasks, verifies outcomes with code-based checkers, and assesses empirical skill dependence through contrastive executions. We construct and release 2,756 environments across 12 categories and collect 8,364 successful trajectories from multiple models and harnesses, averaging 49 tool calls and over 60k logged text tokens. These resources support supervised fine-tuning on verified workflows and reinforcement learning with outcome-based rewards. Under Claude Code, supervised fine-tuning improves Qwen3.5-35B-A3B by 199 Elo on GDPval-AA v2, 19.10 percentage points on Terminal-Bench 2.1, and 28.13 and 12.38 points on SkillsBench v1.1 with and without skills, respectively. Our 35B textttSkillGym-Agent reaches 51.47% on skill-assisted SkillsBench, exceeding reported scores for Claude Sonnet 4.6, GPT-5.4 Mini, and DeepSeek V4 Pro. Without skills, it also surpasses skill-assisted bases under Codex and Claude Code, suggesting reusable procedural competence.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 23 Sep 2026.
Builds on
28 of 41 references.
References · 28
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
16 more
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
- 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 393
- 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
- 2022TALM: Tool Augmented Language Modelscited by 197
- 2026On Data Engineering for Scaling LLM Terminal Capabilitiescited by 43
- 2026Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environmentscited by 33
- 2026Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agentcited by 22
- 2026OpenThoughts-Agent: Data Recipes for Agentic Modelscited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation
- 2026SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories
- 2026SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement
- 2026SkillFM: Generating Skills for LLM Agents via Latent Flow Matching
- 2026SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents
- 2026SkillMaster: Toward Autonomous Skill Mastery in LLM Agents
- 2026A Framework for Evaluating Agentic Skills at Scale
- 2026SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation