mlsanityData to 7 Oct 2026

Who Maintains Agent Skills? A Longitudinal Study of Human-Governed, AI-Assisted Skill Maintenance

arXiv 2609.05677 ↗First posted 4 Sep 2026cs.CL cs.AI cs.HCcited by 0

On the map: LLM agents and tool use → Agent skill learning and evolution

Abstract (arXiv)

Lifelong LLM agents increasingly rely on external skill artifacts as one element for preserving and reusing capabilities over time. These skills (usually portable Markdown files such as SKILL.md) describe when and how to apply a capability and must be corrected, expanded, and consolidated as tools and usage patterns shift over deployment. Recent work seeks to automate skill curation, but it largely evaluates against automated baselines and treats human maintenance as an unmeasured bottleneck. We study that missing process directly. We mine the full commit histories of five public AI-skill repositories, a purposive sample of AI-tooling organizations, covering 873 commits, 143 skill files, and 254 substantive post-creation edits from October 2025 to June 2026. We code each edit with pre-registered governance, operation, and trigger-evidence codebooks. Three findings emerge. First, every substantive edit is authored or merged through a named human account, while 62% carry an AI co-author trailer, with large repository-level variation. Second, these edits are genuine curation: an audited sample shows that most change skill content, and the coded operations are dominated by additions and corrections. Third, a pre-registered rule-likeness axis fails its reliability gate; reliably coding rule-likeness from commit artifacts remains an open measurement problem. We release the corpus, codebooks, mining scripts, and a replay protocol for automated skill curators. For self-evolving agents, public skill maintenance currently looks less like an autonomous pipeline than a human-governed, AI-assisted loop that future curators must measure against and operate within.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 31 of its 36 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

31 of 36 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026SkillHone: A Harness for Continual Agent Skill Evolution Through Persistent Decision History7 Jun
  2. 2026SkillAdaptor: Self-Adapting Skills for LLM Agents from Trajectories31 May
  3. 2026MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation26 May
  4. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skills22 May
  5. 2026More Skills, Worse Agents? Skill Shadowing Degrades Performance When Expanding Skill Libraries21 May
  6. 2026SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphs12 May
  7. 2026SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Tasks22 Apr
  8. 2026How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings6 Apr
  9. 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills26 Mar
  10. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks13 Feb
  11. 2026Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality8 Feb

Other references · 20

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  3. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  4. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  5. 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
  6. 2026SKILLFOUNDRY: Building Self-Evolving Agent Skill Libraries from Heterogeneous Scientific Resourcescited by 38
  7. 2025Agent READMEs: An Empirical Study of Context Files for Agentic Codingcited by 21
  8. 2024Prompting in the Wild: An Empirical Study of Prompt Evolution in Software Repositoriescited by 16
  9. 2025Context Engineering for AI Agents in Open-Source Softwarecited by 14
  10. 2025Learning from Supervision with Semantic and Episodic Memory: A Reflective Approach to Agent Adaptationcited by 6
  11. 2026Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activitycited by 6
  12. 2025WINELL: Wikipedia Never-Ending Updating with LLM Agentscited by 5
8 more
  1. 2026An Empirical Study of the Evolution of GitHub Actions Workflowscited by 3
  2. 2026A Dataset of Agentic AI Coding Tool Configurationscited by 3
  3. 2026ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agentscited by 2
  4. 2025The AI Attribution Paradox: Transparency as Social Strategy in Open-Source Software Developmentcited by 1
  5. 2026Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Studycited by 1
  6. 2026Mining Architectural Quality Under Agentic AI Adoption: A Causal Study of Java Repositoriescited by 1
  7. 2026Beyond Simpson's Paradox: A Cascade of Confounders in AI Agent Pull-Request Co-Authorshipcited by 1
  8. 2026How Do Developers Maintain and Evolve Their Agents' Instructions? An Empirical Studycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintained
  2. 2026From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills
  3. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries
  4. 2026SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents
  5. 2026What Keeps Agent Skills from Being Reusable? Evidence from 138K SKILL.md Files
  6. 2026Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?
  7. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agents
  8. 2026SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?