# Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?

arXiv: https://arxiv.org/abs/2608.04828 · HTML: https://mlsanity.com/paper/2608.04828/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Placed in

- LLM agents and tool use / [Agent skill learning and evolution](https://mlsanity.com/agents/agent-skill-learning-and-evolution/)

## Builds on (most cited first, 15 of 15)

- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks](https://mlsanity.com/paper/2602.12670/) (2026)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution](https://mlsanity.com/paper/2603.01145/) (2026)
- [Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following](https://mlsanity.com/paper/2410.15553/) (2024)
- [Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality](https://mlsanity.com/paper/2602.08004/) (2026)
- [Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale](https://mlsanity.com/paper/2603.02176/) (2026)
- [Memento-Skills: Let Agents Design Agents](https://mlsanity.com/paper/2603.18743/) (2026)
- [Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows](https://mlsanity.com/paper/2605.27922/) (2026)
- [SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks](https://mlsanity.com/paper/2604.20087/) (2026)
- [SkillNet: Create, Evaluate, and Connect AI Skills](https://mlsanity.com/paper/2603.04448/) (2026)
- [Stop Comparing LLM Agents Without Disclosing the Harness](https://mlsanity.com/paper/2605.23950/) (2026)
- [SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints](https://mlsanity.com/paper/2503.08669/) (2025)
- [Skill-R1: Agent Skill Evolution via Reinforcement Learning](https://mlsanity.com/paper/2605.09359/) (2026)
- [SLBench: Evaluating How LLM Agents Follow Logical Relations in Skills](https://mlsanity.com/paper/2607.09016/) (2026)

## Built on by (most cited first, 10 of 10)

- [Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence](https://mlsanity.com/paper/2608.21156/) (2026)
- [SkillGate: Training In-Policy Skill Selection in Long-Horizon Agents](https://mlsanity.com/paper/2608.18852/) (2026)
- [From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents](https://mlsanity.com/paper/2609.04869/) (2026)
- [The Router Within: Eliciting Native Skill Routing from a Frozen LLM](https://mlsanity.com/paper/2609.15982/) (2026)
- [SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation](https://mlsanity.com/paper/2609.37539/) (2026)
- [An Empirical Study of Agent Skills' Downstream Utility](https://mlsanity.com/paper/2610.08875/) (2026)
- [One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents](https://mlsanity.com/paper/2610.11647/) (2026)
- [Cortex: Measurement-Gated Evolution and Pruning of Coding-Agent Skills and Rules](https://mlsanity.com/paper/2610.14216/) (2026)
- [SkillMarker: A KGW-Based Watermarking Method for Agent Skill Ownership Verification](https://mlsanity.com/paper/2610.15725/) (2026)
- [Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents](https://mlsanity.com/paper/2610.20344/) (2026)

## Abstract (arXiv)

Large language model (LLM) agents increasingly rely on skills, structured documents that specify when to act, which procedure to follow, and which tools are allowed. Existing evaluations mostly judge the quality of a skill or its contribution to task success, leaving unexamined whether an agent can recognize a relevant skill and apply it on its own. We introduce Skill-Use, a benchmark that evaluates skill use under progressive disclosure, where an agent sees only a skill's name and short description and must retrieve the full procedure before following it. Skill-Use separates three facets of skill use. Trigger measures whether the agent invokes the relevant skill, Compliance measures how faithfully it follows the prescribed procedure, and Boundary measures whether it avoids forbidden operations. A Skill-Use (SU) score combines the three and credits execution only after the skill is triggered. Skill-Use pairs 79 real skills with 177 executable tasks across nine domains, each grounded in real files, run in an isolated Docker sandbox, and scored by a trajectory-based rubric. Evaluating eight LLMs under two agent harnesses, we find that reliable skill use remains out of reach, as the strongest configuration reaches an SU of only 0.613. Triggering and procedural compliance fail as independent bottlenecks, and both scores and model rankings shift with the harness, so skill use behaves as a capability conditioned on the harness rather than a fixed property of the model.