# An Empirical Study of Agent Skills' Downstream Utility arXiv: https://arxiv.org/abs/2610.08875 · HTML: https://mlsanity.com/paper/2610.08875/ ## Key facts - No citing papers in the map yet (data through 14 Oct 2026). - Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. - Builds on 26 of its 59 references that are in the map. - First posted on arXiv 6 Oct 2026. ## Placed in - LLM agents and tool use / [Agent skill learning and evolution](https://mlsanity.com/agents/agent-skill-learning-and-evolution/) ## Builds on (most cited first, 26 of 26) - [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023) - [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024) - [SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks](https://mlsanity.com/paper/2602.12670/) (2026) - [SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning](https://mlsanity.com/paper/2602.08234/) (2026) - [Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale](https://mlsanity.com/paper/2601.10338/) (2026) - [SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?](https://mlsanity.com/paper/2603.15401/) (2026) - [Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality](https://mlsanity.com/paper/2602.08004/) (2026) - [Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale](https://mlsanity.com/paper/2603.02176/) (2026) - [SkillRouter: Skill Routing for LLM Agents at Scale](https://mlsanity.com/paper/2603.22455/) (2026) - [Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills](https://mlsanity.com/paper/2604.05333/) (2026) - [SkillNet: Create, Evaluate, and Connect AI Skills](https://mlsanity.com/paper/2603.04448/) (2026) - [Skill Retrieval Augmentation for Agentic AI](https://mlsanity.com/paper/2604.24594/) (2026) - [SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents](https://mlsanity.com/paper/2605.05726/) (2026) - [SkillReducer: Optimizing LLM Agent Skills for Token Efficiency](https://mlsanity.com/paper/2603.29919/) (2026) - [GraSP: Graph-Structured Skill Compositions for LLM Agents](https://mlsanity.com/paper/2604.17870/) (2026) - [From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills](https://mlsanity.com/paper/2605.23899/) (2026) - [SkVM: Revisiting Language VM for Skills across Heterogenous LLMs and Harnesses](https://mlsanity.com/paper/2604.03088/) (2026) - [From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skills](https://mlsanity.com/paper/2604.24026/) (2026) - [SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineering](https://mlsanity.com/paper/2604.09297/) (2026) - [Group of Skills: Group-Structured Skill Retrieval for Agent Skill Libraries](https://mlsanity.com/paper/2605.06978/) (2026) - [Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?](https://mlsanity.com/paper/2608.04828/) (2026) - [SkCC: Portable and Secure Skill Compilation for Cross-Framework LLM Agents](https://mlsanity.com/paper/2605.03353/) (2026) - [Not All Skills Help: Measuring and Repairing Agent Knowledge](https://mlsanity.com/paper/2606.15390/) (2026) - [From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills](https://mlsanity.com/paper/2607.01456/) (2026) - [Do LLM-Generated Skills Make Better AI Data Scientists? A Component Ablation Across Data-Science Workflows](https://mlsanity.com/paper/2607.07504/) (2026) - [An Empirical Study of Downstream Adaptation for Agent Skills](https://mlsanity.com/paper/2607.03238/) (2026) ## Abstract (arXiv) Agent Skills package procedural guidance and resources for reuse, but a relevant Skill does not necessarily improve task performance. Existing studies characterize Skill content and evaluate downstream performance, yet provide limited explanations of how utility depends on content, execution configuration, and multi-Skill organization. We conduct an empirical study on 87 SkillsBench tasks, defining downstream utility as the pass-rate difference from No-Skill on the same tasks under the same model--harness configuration. We compare the same Skills across nine configurations, then examine alternative published Skills and organizations of fixed Skill sets under three selected configurations. We retrieve marketplace candidates from a curated corpus of 37,596 Skills. LLM-assisted analysis of content, execution traces, and final artifacts, followed by author review, relates provided support to actual use and task outcomes. The same Skills help some configurations and hurt others on 36.78% of tasks, with trajectories showing that recommended procedures can become an execution burden. Relevance rankings overlook more useful candidates. Within the evaluated candidate sets, reranking by support for required operations raises first-choice pass rates by 4.35--5.80 percentage points across the three configurations. We derive 17 authoring practices linking executable procedures to recovery, preservation of task requirements, and checks on final artifacts. Stage Plan and Dependency DAG outperform use order alone, with DAG's additional benefits concentrated in tasks supplied with five or six Skills. These findings guide developers to assess usable operation support, allow procedure adaptation while preserving task requirements, and make artifact dependencies explicit when organizing Skills.