SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing

Agent skills provide LLM agents with reusable procedures, but a given skill may be incomplete, ineffective, or misleading for the current task. Existing methods for revising skills typically rely on external outcome supervision, such as task rewards, performance on a separate validation set, or feedback from deployment, to decide which updates to retain. However, this supervision may be costly or unavailable when adapting a skill to a new task. Using execution behavior as an alternative signal is challenging: a single run reflects the combined effects of the agent, task, and workspace, making the skill's contribution difficult to isolate. Comparing executions with and without the skill, while holding these factors fixed, provides evidence about how the skill changes behavior. We introduce SelfEvoSkill, which links these behavioral differences to relevant skill passages and iteratively revises and reevaluates those passages through subsequent executions. Checks derived once from the task's explicit requirements prevent revisions from violating task constraints. Across 89 containerized tasks in 8 professional domains, under the same best-of-two evaluation budget, SelfEvoSkill attains a 73.9% mean task reward, exceeding the no-skill condition by +33.0 percentage points and the benchmark's curated skill by +17.2 percentage points, while skill evolution receives neither task rewards nor benchmark verifier outputs. In controlled ablations, using two with-skill runs instead of a matched with/without-skill pair reduces mean reward by 4.6 percentage points, while limiting revision to one round reduces it by 9.0 points relative to repeated revision. Together, these results show that behavioral differences between matched executions can provide an effective signal for revising skills when outcome supervision is unavailable.