From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills
On the map: LLM agents and tool use → Agent skill learning and evolution
Abstract (arXiv)
Agent Skills provide on-demand domain knowledge to LLM agents without requiring model retraining. Each Agent Skill is defined by a mandatory SKILL.md file containing metadata and an unstructured Markdown body whose contents are left entirely to the skill author. Despite the rapid adoption of Agent Skills, little is known about how these files are authored or whether existing authoring guidelines are followed in practice. In this paper, we present the first systematic study of SKILL.md files as a software artifact. We qualitatively analyze 238 real-world skills and derive a taxonomy of 13 higher-level and 44 lower-level semantic components. We then conduct a multivocal literature review of 29 sources to identify best practices for authoring SKILL.md files and introduce skill smells as violations of these practices. Finally, we develop an automated detector and apply it to real-world skills, finding that over 99% of SKILL.md files contain at least one skill smell, and once introduced, skill smells rarely disappear as skills evolve. These findings reveal a substantial gap between recommended and actual authoring practices, motivating the development of automated techniques to remediate skill smells while increasing developer awareness of this emerging quality issue.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 39 of its 69 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
39 of 69 references.
Closest ancestors · 6
Same direction, newest first
- 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolver9 Apr
- 2026SkVM: Revisiting Language VM for Skills across Heterogenous LLMs and Harnesses3 Apr
- 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification2 Apr
- 2026SkillReducer: Optimizing LLM Agent Skills for Token Efficiency31 Mar
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks13 Feb
- 2026Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality8 Feb
Other references · 33
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
- 2016Bag of Tricks for Efficient Text Classificationcited by 869
- 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
- 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
- 2016FastText.zip: Compressing text classification modelscited by 331
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
- 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 150
- 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scalecited by 94
- 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Modelscited by 90
21 more
- 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
- 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
- 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
- 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 40
- 2023Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Modelscited by 37
- 2025What Prompts Don't Say: Understanding and Managing Underspecification in LLM Promptscited by 32
- 2025Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injectionscited by 30
- 2024Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Modelscited by 28
- 2026Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Usecited by 25
- 2025ToolTweak: An Attack on Tool Selection in LLM-based Agentscited by 22
- 2026SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaborationcited by 21
- 2021Sampling Projects in GitHub for MSR Studiescited by 20
- 2025From Prompts to Templates: A Systematic Prompt Template Analysis for Real-world LLMappscited by 19
- 2018Categorizing the Content of GitHub README Filescited by 18
- 2024Prompting in the Wild: An Empirical Study of Prompt Evolution in Software Repositoriescited by 16
- 2026SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineeringcited by 14
- 2025Procedural Knowledge Improves Agentic LLM Workflowscited by 7
- 2024Context Conquers Parameters: Outperforming Proprietary LLM in Commit Message Generationcited by 6
- 2024What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworkscited by 4
- 2024Recognizing Limits: Investigating Infeasibility in Large Language Modelscited by 4
- 2025Larger Is Not Always Better: Exploring Small Open-source Language Models in Logging Statement Generationcited by 3
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Skillware: A Software Ontology and Engineering Lifecycle for Persistent Behavioral Artifactscited by 1
- 2026Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skillscited by 1
- 2026mimeo: Compiling Public Expert Corpora into Agent Skills and Testing What Transferscited by 0
- 2026Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflowscited by 0
- 2026Trajectory-Guided Fault Localization for Agent Skill Evolutioncited by 0
- 2026Trajectory-Guided Fault Localization for Agent Skill Evolution8 Oct 2026
- 2026Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflows23 Sep 2026
- 2026Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills2 Sep 2026
- 2026mimeo: Compiling Public Expert Corpora into Agent Skills and Testing What Transfers31 Aug 2026
- 2026Skillware: A Software Ontology and Engineering Lifecycle for Persistent Behavioral Artifacts21 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026A Framework for Evaluating Agentic Skills at Scale
- 2026SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents
- 2026How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings
- 2026Authoring Agent Skills: A Software-Engineering Approach
- 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintained
- 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries
- 2026Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?
- 2026SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior