SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
External skills extend the capabilities of large language model agents, but also introduce an execution-time attack surface: a skill that appears benign under inspection may reveal harmful behavior only after particular environmental states, resources, or interaction histories are encountered. Existing scanners primarily rely on static analysis, predefined rules, or one-shot semantic judgments, making such conditional behavior difficult to elicit and attribute. We present SkillSentry, a dynamic safety-testing framework based on adaptive honey worlds. SkillSentry infers the intended capability boundary of a skill, constructs an LLM-simulated environment with controlled decoy resources, and adaptively generates tasks to explore its behavioral states. It then compares skill-enabled trajectories with matched no-skill executions, grounding suspicious behaviors in source code and verified execution traces before making a final decision. We evaluate SkillSentry against seven scanner configurations. SkillSentry achieves 99.50% Recall and 96.26% average F1 on standard benchmarks. Under semantics-preserving evasion, it reaches 92.95% average F1, compared with 80.07% for the strongest baselines. Our code is available at https://github.com/nizhangli062-jpg/SkillSentry-Adaptive-Honey-Worlds-for-Dynamic-Safety-Testing-of-Agent-Skills.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 13 of its 28 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
13 of 28 references.
Closest ancestors · 11
Same direction, newest first
- 2026Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification2 Jul
- 2026Detecting Malicious Agent Skills in the Wild using Attention22 Jun
- 2026PhantomSkill: Malicious Code Injection in Agent Skill Ecosystems17 Jun
- 2026SkillMutator: Benchmarking and Defending Language-and-Code Cross-modal Attacks on LLM Agent Skills12 Jun
- 2026Poise: Position-Aware One-Instruction Skill Injection for Silent Execution on LLM Agents6 Jun
- 2026MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills5 Jun
- 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks23 Feb
- 2026SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents15 Feb
- 2026"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild6 Feb
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
Other references · 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SkillAtlas: An Attack Trace Library for Agent Skills
- 2026SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces
- 2026Agent Skill Security: Threat Models, Attacks, Defenses, and Evaluation
- 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries
- 2026Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security
- 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
- 2026Towards a Risk Assessment of Malicious Skill Files in Coding Agents
- 2026SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction