mlsanityData to 7 Oct 2026

From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills

arXiv 2607.01456 ↗First posted 1 Jul 2026cs.SEcited by 5

On the map: LLM agents and tool use → Agent skill learning and evolution

Abstract (arXiv)

Agent Skills provide on-demand domain knowledge to LLM agents without requiring model retraining. Each Agent Skill is defined by a mandatory SKILL.md file containing metadata and an unstructured Markdown body whose contents are left entirely to the skill author. Despite the rapid adoption of Agent Skills, little is known about how these files are authored or whether existing authoring guidelines are followed in practice. In this paper, we present the first systematic study of SKILL.md files as a software artifact. We qualitatively analyze 238 real-world skills and derive a taxonomy of 13 higher-level and 44 lower-level semantic components. We then conduct a multivocal literature review of 29 sources to identify best practices for authoring SKILL.md files and introduce skill smells as violations of these practices. Finally, we develop an automated detector and apply it to real-world skills, finding that over 99% of SKILL.md files contain at least one skill smell, and once introduced, skill smells rarely disappear as skills evolve. These findings reveal a substantial gap between recommended and actual authoring practices, motivating the development of automated techniques to remediate skill smells while increasing developer awareness of this emerging quality issue.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 39 of its 69 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

39 of 69 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolver9 Apr
  2. 2026SkVM: Revisiting Language VM for Skills across Heterogenous LLMs and Harnesses3 Apr
  3. 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification2 Apr
  4. 2026SkillReducer: Optimizing LLM Agent Skills for Token Efficiency31 Mar
  5. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks13 Feb
  6. 2026Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality8 Feb

Other references · 33

  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  3. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  5. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  7. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  8. 2016FastText.zip: Compressing text classification modelscited by 331
  9. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
  10. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 150
  11. 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scalecited by 94
  12. 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Modelscited by 90
21 more
  1. 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
  2. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
  3. 2025Web-Shepherd: Advancing PRMs for Reinforcing Web Agentscited by 41
  4. 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 40
  5. 2023Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Modelscited by 37
  6. 2025What Prompts Don't Say: Understanding and Managing Underspecification in LLM Promptscited by 32
  7. 2025Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injectionscited by 30
  8. 2024Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Modelscited by 28
  9. 2026Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Usecited by 25
  10. 2025ToolTweak: An Attack on Tool Selection in LLM-based Agentscited by 22
  11. 2026SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaborationcited by 21
  12. 2021Sampling Projects in GitHub for MSR Studiescited by 20
  13. 2025From Prompts to Templates: A Systematic Prompt Template Analysis for Real-world LLMappscited by 19
  14. 2018Categorizing the Content of GitHub README Filescited by 18
  15. 2024Prompting in the Wild: An Empirical Study of Prompt Evolution in Software Repositoriescited by 16
  16. 2026SkillMOO: Multi-Objective Optimization of Agent Skills for Software Engineeringcited by 14
  17. 2025Procedural Knowledge Improves Agentic LLM Workflowscited by 7
  18. 2024Context Conquers Parameters: Outperforming Proprietary LLM in Commit Message Generationcited by 6
  19. 2024What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworkscited by 4
  20. 2024Recognizing Limits: Investigating Infeasibility in Large Language Modelscited by 4
  21. 2025Larger Is Not Always Better: Exploring Small Open-source Language Models in Logging Statement Generationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Framework for Evaluating Agentic Skills at Scale
  2. 2026SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents
  3. 2026How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings
  4. 2026Authoring Agent Skills: A Software-Engineering Approach
  5. 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintained
  6. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries
  7. 2026Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?
  8. 2026SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior