Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents
Not on the map. Why?
Abstract (arXiv)
Skill-augmented agents load reusable skills as persistent runtime context, improving task performance but also giving malicious skills a durable channel for steering future actions. Such skills may leak secrets, corrupt code, bypass approvals, or stage data for exfiltration only after a concrete user task and workspace state make the unsafe action appear useful. This makes pre-install vetting insufficient and calls for runtime, task-conditioned protection. We propose Defense-as-Skill, a defense paradigm that implements the runtime guard itself as an installable, inspectable, and editable skill. Our guard, SkillSonar, runs alongside untrusted task skills and checks sensitive actions against the user's task boundary, routing each action to an allow, replan, or confirmation decision without modifying the underlying agent runtime. To study this setting, we construct SCOPE-R, a task-conditioned dataset covering 6 risk families and 21 sub-categories, with 206 attack-confirmed malicious instances and 43 benign tasks. We then improve SkillSonar on the SCOPE-R training subset using runtime guard-skill evolution, a Monte-Carlo Tree Search procedure that evolves the on-disk guard skill from feedback on the rollouts. Across Claude Code and OpenClaw, the evolved guard substantially reduces attack success while maintaining a favorable safety-utility trade-off; on repeated GLM-5 runs, ID ASR drops from 0.482 to 0.104 and OOD ASR from 0.606 to 0.115, with gains transferring across victim models, held-out risk families, and external benchmarks. These results show that runtime guards can be skill-native, transparent, and evolvable policy artifacts for skill-augmented agents.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 30 references that are in the map. First posted on arXiv 1 Sep 2026.
Builds on
19 of 30 references.
References · 19
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
- 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 223
- 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verificationcited by 128
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 113
- 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scalecited by 105
- 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 90
- 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 81
- 2026SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Taskscited by 55
- 2026"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wildcited by 50
- 2026WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluationcited by 37
7 more
- 2026SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinementcited by 30
- 2026Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threatscited by 29
- 2026SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agentscited by 25
- 2026Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClawcited by 20
- 2026SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfacescited by 19
- 2024Agent Skill Acquisition for Large Language Models via CycleQDcited by 12
- 2026SkillPyramid: A Hierarchical Skill Consolidation Framework for Self-Evolving Agentscited by 4
Built on by
Not cited yet.