Safe Skill Retirement for Physical Agents
Not on the map. Why?
Abstract (arXiv)
Agent skills bundle procedural guidance with execution conditions governing authority, user consent, and live environment state. When model capabilities advance, maintainers prune instructions that appear redundant on authorized benchmark tasks. However, authorized maintenance tests can leave dormant safety conditions untested. This mismatch creates an unmeasured support gap over physical and privacy-sensitive effects. We introduce matched authority counterfactuals that hold the requested action, tool parameters, and intended effect fixed while systematically varying a single governing predicate. We formalize this evaluation via a two-gate retirement certificate requiring a candidate reduction to preserve authorized utility within a declared margin while producing zero unauthorized protected effects. In controlled experiments spanning four frontier and local model configurations across twelve skill bundles (2,592 evaluation cells), task-certified reductions remove over 94% of skill clauses and preserve authorized completion, yet produce unauthorized protected effects in every bundle. Boundary enforcement eliminates protected effects on the declared audit but fails the utility gate for one configuration. One bounded combined protocol passes both gates across all four configurations, with zero utility headroom. An end-to-end check on one read-only Home Assistant camera chain verifies proposal, decision, and effect measurements on a real device. These results demonstrate that while task benchmarks can justify retiring procedural guidance, retirement decisions require explicitly auditing the authority contracts governing physical actions.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 25 Aug 2026.
Builds on
19 of 26 references.
References · 19
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
- 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
- 2025Defeating Prompt Injections by Designcited by 297
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 223
- 2025Progent: Securing AI Agents with Privilege Controlcited by 141
- 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scalecited by 105
- 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 90
- 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 82
- 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedbackcited by 72
7 more
- 2026SkillReducer: Optimizing LLM Agent Skills for Token Efficiencycited by 33
- 2023LLMind: Orchestrating AI and IoT with LLM for Complex Task Executioncited by 24
- 2025Subtle Risks, Critical Failures: A Framework for Diagnosing Physical Safety of LLMs for Embodied Decision Makingcited by 20
- 2026ToolGate: Contract-Grounded and Verified Tool Execution for LLMscited by 15
- 2023SAGE: Smart home Agent with Grounded Executioncited by 13
- 2026SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinementcited by 12
- 2026Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agentscited by 10
Built on by
Citing papers per year: 2026: 1.
1 paper.