When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
LLM agents increasingly rely on community-contributed skills that expand an agent's operational capability set. We study a core safety problem in agentic AI systems: whether individually safe skills can compose into unsafe installed skill sets. We present SkillReact, a compositional security measurement framework with three components: a deterministic static-composition benchmark, a two-rater LLM-assisted human-adjudication pipeline, and an action-based exploitability harness. On 1,520 ClawHub skills, 651 pass individual inspection and form 211,575 pairs; the benchmark flags 22.25% of these as structural candidates. We treat this raw rate as a recall-oriented scanner ceiling and calibrate it against human judgment: in a pattern-stratified audit, roughly one in five flagged pair-pattern hits survives as a real compositional risk (population-weighted validity 18.2%, our headline result), implying about 14K genuine risk memberships in a single registry that per-skill scanning misses by construction, since every pair is individually safe. An action-based harness then probes when these candidates become model-issued tool calls, and finds realization gated by host-model disposition: on an anchor-conditioned dropper subset, Haiku-4-5 issues the dropper-stage tool call on all 39 direct-prompt trials (36 of them the full download-then-execute chain, 3 download-only), Opus-4-7 stops at the download, and Sonnet-4-6 refuses outright. A control that holds the request fixed and varies only the installed skills finds compliance highest with no skills installed: a composition fixes which capabilities are reachable, while the host model decides whether to use them. Together these motivate install-time compositional checks and capability isolation as complements to per-skill scanning.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 36 of its 55 references that are in the map. First posted on arXiv 30 May 2026.
Builds on
36 of 55 references.
Closest ancestors · 19
Same direction, newest first
- 2026SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills8 Apr
- 2026Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems3 Apr
- 2026How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study3 Apr
- 2026A Security Analysis of the OpenClaw AI Agent Framework29 Mar
- 2026Formal Analysis and Supply Chain Security for Agentic AI Skills27 Feb
- 2026SOK: A Taxonomy of Attack Vectors and Defense Strategies for Agentic Supply Chain Runtime23 Feb
- 2026SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents15 Feb
- 2026"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild6 Feb
- 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems24 Jan
- 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale15 Jan
- 2025Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation18 Dec
- 2025MindGuard: Intrinsic Decision Inspection for Securing LLM Agents Against Metadata Poisoning28 Aug
7 more
- 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
- 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
- 2025Progent: Securing AI Agents with Privilege Control16 Apr
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
- 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
Other references · 17
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
- 2025DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Modelscited by 29
- 2026BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoningcited by 25
- 2025Does Tone Change the Answer? Evaluating Prompt Politeness Effects on Modern LLMs: GPT, Gemini, and LLaMAcited by 23
- 2026SCRIBE: Structured Mid-Level Supervision for Tool-Using Language Modelscited by 20
- 2026Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysiscited by 20
- 2025Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoningcited by 15
- 2026Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Modelscited by 15
- 2026Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-offcited by 13
- 2026Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflectioncited by 12
- 2026Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillationcited by 11
- 2026AI for Auto-Research: Roadmap & User Guidecited by 11
5 more
- 2026ChainCaps: Composition-Safe Tool-Using Agents via Monotonic Capability Attenuationcited by 11
- 2025Reward Auditor: Inference on Reward Modeling Suitability in Real-World Perturbed Scenarioscited by 7
- 2025Alleviating Attention Hacking in Discriminative Reward Modeling through Interaction Distillationcited by 5
- 2026Safety is Non-Compositional: A Formal Framework for Capability-Based AI Systemscited by 4
- 2026From Component Manipulation to System Compromise: Understanding and Detecting Malicious MCP Serverscited by 4
Built on by
Citing papers per year: 2026: 14.
14 papers.
Sort:
By area:
- 2026Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluationcited by 8
- 2026DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderationcited by 4
- 2026ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenancecited by 2
- 2026Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Accesscited by 2
- 2026CompoSkill: Compositional Skill Chain Attacks from Individually Scanner-Passing LLM Agent Skillscited by 2
- 2026Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elitecited by 1
- 2026Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivitycited by 1
- 2026MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignmentcited by 0
- 2026Governing Generative AI Across Financial Institutions: A Framework for Generative AI Risk Controlcited by 0
- 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detectioncited by 0
- 2026Bioinfoysis Technical Reportcited by 0
- 2026Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdictscited by 0
- 2026One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents8 Oct 2026
- 2026Hiding in Plain Sight: Decoupling Pretext from Actuation for Skill Poisoning in LLM Agents30 Sep 2026
- 2026Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts8 Sep 2026
- 2026Bioinfoysis Technical Report3 Sep 2026
- 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection20 Aug 2026
- 2026CompoSkill: Compositional Skill Chain Attacks from Individually Scanner-Passing LLM Agent Skills17 Aug 2026
- 2026Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Access9 Aug 2026
- 2026Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity1 Aug 2026
- 2026Governing Generative AI Across Financial Institutions: A Framework for Generative AI Risk Control5 Jul 2026
- 2026ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance1 Jul 2026
- 2026MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignment27 Jun 2026
- 2026DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderation27 Jun 2026
- 2026ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenancecited by 2
- 2026CompoSkill: Compositional Skill Chain Attacks from Individually Scanner-Passing LLM Agent Skillscited by 2
- 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detectioncited by 0
- 2026Bioinfoysis Technical Reportcited by 0
- 2026Hiding in Plain Sight: Decoupling Pretext from Actuation for Skill Poisoning in LLM Agentscited by 0
- 2026Hiding in Plain Sight: Decoupling Pretext from Actuation for Skill Poisoning in LLM Agents30 Sep 2026
- 2026Bioinfoysis Technical Report3 Sep 2026
- 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection20 Aug 2026
- 2026CompoSkill: Compositional Skill Chain Attacks from Individually Scanner-Passing LLM Agent Skills17 Aug 2026
- 2026ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance1 Jul 2026
- 2026DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderationcited by 4
- 2026Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Accesscited by 2
- 2026MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignmentcited by 0
- 2026Governing Generative AI Across Financial Institutions: A Framework for Generative AI Risk Controlcited by 0
- 2026One Skill Too Many: How Co-Installed Skills Conflict in Coding Agentscited by 0
- 2026One Skill Too Many: How Co-Installed Skills Conflict in Coding Agents8 Oct 2026
- 2026Same Question, Different Answer? Measuring and Mitigating Prompt Privilege for Equitable AI Access9 Aug 2026
- 2026Governing Generative AI Across Financial Institutions: A Framework for Generative AI Risk Control5 Jul 2026
- 2026MOSAIC: Orchestrating Collaborative Knowledge Tracing with Hierarchical Semantic Alignment27 Jun 2026
- 2026DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderation27 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces
- 2026OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills
- 2026Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents
- 2026SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction
- 2026Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems
- 2026Can Agents Trust Their Skills? Uncovering Unsafe Chains of Trust in Skill-Based LLM Agents
- 2026SkillVetBench: LLM-as-Judge for Multi-Dimensional Security Risk Evaluation in Open-Source LLM Agent Skills
- 2026STARS: Skill-Triggered Audit for Request-Conditioned Invocation Safety in Agent Systems