# SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement

arXiv: https://arxiv.org/abs/2604.04989 · HTML: https://mlsanity.com/paper/2604.04989/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 5 Apr 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 22 of 22)

- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts](https://mlsanity.com/paper/2402.16822/) (2024)
- [Prompt Injection Attack to Tool Selection in LLM Agents](https://mlsanity.com/paper/2504.19793/) (2025)
- [AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents](https://mlsanity.com/paper/2406.13352/) (2024)
- [Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale](https://mlsanity.com/paper/2601.10338/) (2026)
- [Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks](https://mlsanity.com/paper/2602.20156/) (2026)
- [Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks](https://mlsanity.com/paper/2304.14732/) (2023)
- [Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality](https://mlsanity.com/paper/2602.08004/) (2026)
- [Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale](https://mlsanity.com/paper/2603.02176/) (2026)
- ["Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild](https://mlsanity.com/paper/2602.06547/) (2026)
- [Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections](https://mlsanity.com/paper/2510.26328/) (2025)
- [Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems](https://mlsanity.com/paper/2601.17548/) (2026)
- [Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats](https://mlsanity.com/paper/2603.11619/) (2026)
- [from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors](https://mlsanity.com/paper/2503.00038/) (2025)
- [Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs](https://mlsanity.com/paper/2508.16347/) (2025)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [Projecting Out the Malice: A Global Subspace Approach to LLM Detoxification](https://mlsanity.com/paper/2601.06226/) (2026)

## Built on by (most cited first, 30 of 30)

- [SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces](https://mlsanity.com/paper/2605.12015/) (2026)
- [From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skills](https://mlsanity.com/paper/2604.24026/) (2026)
- [Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems](https://mlsanity.com/paper/2606.15242/) (2026)
- [Poise: Position-Aware One-Instruction Skill Injection for Silent Execution on LLM Agents](https://mlsanity.com/paper/2606.07943/) (2026)
- [Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2606.10749/) (2026)
- [SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction](https://mlsanity.com/paper/2606.02540/) (2026)
- [Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries](https://mlsanity.com/paper/2607.10113/) (2026)
- [SkillScope: Toward Fine-Grained Least-Privilege Enforcement for Agent Skills](https://mlsanity.com/paper/2605.05868/) (2026)
- [Runtime Skill Audit: Targeted Runtime Probing for Agent Skill Security](https://mlsanity.com/paper/2606.11671/) (2026)
- [Skills Are Not Islands: Measuring Dependency and Risk in Agent Skill Supply Chains](https://mlsanity.com/paper/2607.01136/) (2026)
- [JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skills](https://mlsanity.com/paper/2608.16465/) (2026)
- [Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClaw](https://mlsanity.com/paper/2605.11047/) (2026)
- [Convergent Detour Hijacking: Task-Preserving Resource Amplification in Skill-Based LLM Agents](https://mlsanity.com/paper/2608.12273/) (2026)
- [CONTRA: Red-Teaming Configurations of Personalizable Agents](https://mlsanity.com/paper/2607.03220/) (2026)
- [RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection](https://mlsanity.com/paper/2608.01630/) (2026)
- [Contractual Skills: A GovernSpec Design Framework for Enterprise AI Agents](https://mlsanity.com/paper/2605.22634/) (2026)
- [Skillsets on the Chain: A Blockchain-based Zero-Trust Framework for Agentic AI Networking](https://mlsanity.com/paper/2608.00104/) (2026)
- [SynChain: Inducing Computer-Use Agent Systems to Construct Their Own Attack Chains](https://mlsanity.com/paper/2608.06862/) (2026)
- [OBLIVION: Workflow-Level Operational Skill Unlearning for Deployed Agents](https://mlsanity.com/paper/2608.08264/) (2026)
- [Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security](https://mlsanity.com/paper/2608.29596/) (2026)
- [Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents](https://mlsanity.com/paper/2609.01487/) (2026)
- [A Finger on the Scale: Covert Policy Steering through Agentic Skills](https://mlsanity.com/paper/2609.02564/) (2026)
- [SkillAtlas: An Attack Trace Library for Agent Skills](https://mlsanity.com/paper/2609.13353/) (2026)
- [SkillDRE: Dual-Stage Red-Team Evolution of Agent Skills via Pre-Execution and Runtime Feedback](https://mlsanity.com/paper/2609.32400/) (2026)
- [Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents](https://mlsanity.com/paper/2609.39607/) (2026)
- [PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents](https://mlsanity.com/paper/2610.01349/) (2026)
- [Runaway Reaction: When Benign Skills Compose into Malicious Behavior](https://mlsanity.com/paper/2610.05943/) (2026)
- [Surviving the Router: Optimizing Skill Injections for Retrieval and Execution](https://mlsanity.com/paper/2610.08098/) (2026)
- [Encrypted Skills Can Become New Security Exploits If AI Agents Know How to Decipher](https://mlsanity.com/paper/2610.18262/) (2026)
- [CoRedSkill: Stealthy Skill Composition Attacks via AST-Guided Injection](https://mlsanity.com/paper/2610.20049/) (2026)

## Abstract (arXiv)

LLM-based agent systems increasingly rely on agent skills sourced from open registries to extend their capabilities, yet the openness of such ecosystems makes skills difficult to thoroughly vet. Existing attacks rely on injecting malicious instructions into skills, making them easily detectable by static auditing. However, non-malicious skills may also harbor latent vulnerabilities that an attacker can exploit solely through adversarial prompting, without modifying the skill itself. We introduce SkillAttack, a red-teaming framework that dynamically verifies skill vulnerability exploitability through adversarial prompting. SkillAttack combines vulnerability analysis, surface-parallel attack generation, and feedback-driven exploit refinement into a closed-loop search that progressively converges toward successful exploitation. Experiments across 10 LLMs on 71 adversarial and 100 real-world skills show that SkillAttack outperforms all baselines by a wide margin (ASR 0.73--0.93 on adversarial skills, up to 0.26 on real-world skills), revealing that even well-intended skills pose serious security risks under realistic agent interactions.