# SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills

arXiv: https://arxiv.org/abs/2604.06550 · HTML: https://mlsanity.com/paper/2604.06550/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 49 references that are in the map.
- First posted on arXiv 8 Apr 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 21 of 21)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models](https://mlsanity.com/paper/2408.08926/) (2024)
- [Prompt Injection Attack to Tool Selection in LLM Agents](https://mlsanity.com/paper/2504.19793/) (2025)
- [Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward](https://mlsanity.com/paper/2602.12430/) (2026)
- [Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale](https://mlsanity.com/paper/2601.10338/) (2026)
- [Teams of LLM Agents can Exploit Zero-Day Vulnerabilities](https://mlsanity.com/paper/2406.01637/) (2024)
- [We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs](https://mlsanity.com/paper/2406.10279/) (2024)
- [Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks](https://mlsanity.com/paper/2005.09535/) (2020)
- [From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows](https://mlsanity.com/paper/2506.23260/) (2025)
- [SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents](https://mlsanity.com/paper/2602.14211/) (2026)
- [Formal Analysis and Supply Chain Security for Agentic AI Skills](https://mlsanity.com/paper/2603.00195/) (2026)
- [SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaboration](https://mlsanity.com/paper/2603.21019/) (2026)
- [SkillTester: Benchmarking Utility and Security of Agent Skills](https://mlsanity.com/paper/2603.28815/) (2026)
- [Agent Audit: A Security Analysis System for LLM Agent Applications](https://mlsanity.com/paper/2603.22853/) (2026)
- [Malware Detection at the Edge with Lightweight LLMs: A Performance Evaluation](https://mlsanity.com/paper/2503.04302/) (2025)
- [Latent Reuse in Agent Skills: Multi-modal Clone Detection at Ecosystem Scale](https://mlsanity.com/paper/2603.22447/) (2026)
- [LoRA-based Parameter-Efficient LLMs for Continuous Learning in Edge-based Malware Detection](https://mlsanity.com/paper/2602.11655/) (2026)

## Built on by (most cited first, 20 of 20)

- [When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems](https://mlsanity.com/paper/2606.00448/) (2026)
- [From Skill Text to Skill Structure: The Scheduling-Structural-Logical Representation for Agent Skills](https://mlsanity.com/paper/2604.24026/) (2026)
- [Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems](https://mlsanity.com/paper/2606.15242/) (2026)
- [Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registry](https://mlsanity.com/paper/2605.11418/) (2026)
- [Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware](https://mlsanity.com/paper/2607.02357/) (2026)
- [Behavioral Integrity Verification for AI Agent Skills](https://mlsanity.com/paper/2605.11770/) (2026)
- [Exploiting LLM Agent Supply Chains via Payload-less Skills](https://mlsanity.com/paper/2605.14460/) (2026)
- [SkillGuard: A Permission-Centric Framework for Agent Skill Security](https://mlsanity.com/paper/2606.03024/) (2026)
- [Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries](https://mlsanity.com/paper/2607.10113/) (2026)
- [ColluSkill: Adversarial Cross-Skill Composition for Evading Agent Skill Scanners](https://mlsanity.com/paper/2608.09732/) (2026)
- [Benchmarking Security Risk Detection and Verification in Open Agentic Skill Ecosystems](https://mlsanity.com/paper/2606.00925/) (2026)
- [SkillConsist: Detecting Inconsistencies in Agent Skills via Bidirectional Graph Alignment](https://mlsanity.com/paper/2608.07639/) (2026)
- [Auto-Policy, not Auto-Skill: Compiled Agent Skills for the Physical World](https://mlsanity.com/paper/2608.25091/) (2026)
- [Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security](https://mlsanity.com/paper/2608.29596/) (2026)
- [SkillSecurer: Detecting and Patching Prompt-Injection Vulnerabilities in AI Agent Skills](https://mlsanity.com/paper/2609.14079/) (2026)
- [SKILLLITE: Evidence-Guided Malicious Skill Auditing with Compact LLMs](https://mlsanity.com/paper/2609.36879/) (2026)
- [Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents](https://mlsanity.com/paper/2609.39607/) (2026)
- [MCRI: A Four-Dimensional Framework for Analyzing and Evaluating Agent Skills](https://mlsanity.com/paper/2610.01506/) (2026)
- [AgentSpy: Making AI Agent Behavior Observable](https://mlsanity.com/paper/2610.06001/) (2026)
- [Surviving the Router: Optimizing Skill Injections for Retrieval and Execution](https://mlsanity.com/paper/2610.08098/) (2026)

## Abstract (arXiv)

Agent skills combine natural-language instructions with executable code while inheriting an agent's filesystem, credential, and network access. Attacks can span prose and files, whereas regex and code-only analyzers cover only one modality. SkillSieve applies three progressively deeper layers: recall-oriented regex, AST, and metadata triage; four parallel LLM security sub-tasks; and an independent three-model jury with debate on disagreement. We evaluate 49,592 real ClawHub skills, a 390-skill labeled benchmark, and 100 adversarial samples across five evasion techniques on a 440 USD ARM board. The full pipeline achieves F1 = 0.929 (precision 0.912, recall 0.945) at an average cost of 0.006 per skill. An optional XGBoost fast path reduces Layer-2/3 calls by 32% with a 1.7-point F1 decrease while preserving recall. On 52 Feishu/Lark packages, Layer 2 reclassifies 13 of 14 Layer-1 flags as safe after contextual analysis; we also deploy the system as a Feishu chat bot. Code, labels, and aggregate results are open-sourced.