mlsanityData to 7 Oct 2026

One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents

arXiv 2610.11647 ↗First posted 8 Oct 2026cs.SE cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Coding agents are extended with agent skills, directories whose SKILL.md tells the model when and how to perform a task. Because skills come from independent sources (teams, developers, plugins, copied collections), an installed skill can be co-installed with a similar skill doing the same job, and the model picks between them by name and description alone. In a conflict, the installed skill loses core functions (e.g., a ban on touching git) because the similar skill runs instead or changes what it does. The task still passes, so benchmarks that check only task completion miss such cases. We present the first empirical study of such conflicts. From snapshots of 20,947 repositories, we mine 822,109 candidate similar-skill pairs, have an LLM judge a stratified sample of 3,754, and run 312 confirmed pairs on three models (6,368 runs, 169,294 tool calls, 542 agent-hours). We report five findings. (1) Conflict-prone skills are common: nearly one in four installed skills is co-installed with one that does the same job, and 37% of judged skills sit inside copied collections. (2) Most such pairs involve normative skills, then capability skills. (3) Without lowering task completion, a similar skill takes one in five runs from the installed skill, and runs that open the similar skill first lose over a third of the exclusive core functions that only the installed skill fulfills. (4) Install location decides which skill runs, listing order barely matters, and the final reply names the skill used in only 0.9% of substituted runs. (5) Conflicts are decided at the first skill read, almost always before any file is changed, and a pre-tool hook at that read restores fidelity on exclusive core functions to the level of runs that open the installed skill first. Benchmarks should thus score exclusive core functions, and platforms should guard the first read and show which skill ran.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 69 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

21 of 69 references.

References · 21

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  3. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  4. 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scalecited by 94
  5. 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
  6. 2024TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generationcited by 62
  7. 2026How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settingscited by 62
  8. 2026SkillRouter: Skill Routing for LLM Agents at Scalecited by 57
  9. 2026SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agentscited by 37
  10. 2025Guidelines for Empirical Studies in Software Engineering involving Large Language Modelscited by 18
  11. 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystemscited by 14
  12. 2026More Skills, Worse Agents? Skill Shadowing Degrades Performance When Expanding Skill Librariescited by 11
9 more
  1. 2026Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registrycited by 10
  2. 2026SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Usecited by 7
  3. 2026A Framework for Evaluating Agentic Skills at Scalecited by 6
  4. 2026Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?cited by 6
  5. 2020Empirical Standards for Software Engineering Researchcited by 4
  6. 2026SLBench: Evaluating How LLM Agents Follow Logical Relations in Skillscited by 2
  7. 2025An Empirical Study of Dotfiles Repositories Containing User-Specific Configuration Filescited by 1
  8. 2026What Keeps Agent Skills from Being Reusable? Evidence from 138K SKILL.md Filescited by 1
  9. 2026Grounded Checklist Partial Credit for Agent Skill Trajectoriescited by 1

Built on by

Not cited yet.