# SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaboration

arXiv: https://arxiv.org/abs/2603.21019 · HTML: https://mlsanity.com/paper/2603.21019/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 22 Mar 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 17 of 17)

- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [Defending Against Indirect Prompt Injection Attacks With Spotlighting](https://mlsanity.com/paper/2403.14720/) (2024)
- [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023)
- [SoK: Agentic Skills -- Beyond Tool Use in LLM Agents](https://mlsanity.com/paper/2602.20867/) (2026)
- [Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale](https://mlsanity.com/paper/2601.10338/) (2026)
- [Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks](https://mlsanity.com/paper/2602.20156/) (2026)
- [Multi-Agent Systems Execute Arbitrary Malicious Code](https://mlsanity.com/paper/2503.12188/) (2025)
- ["Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild](https://mlsanity.com/paper/2602.06547/) (2026)
- [Deep Just-In-Time Inconsistency Detection Between Comments and Source Code](https://mlsanity.com/paper/2010.01625/) (2020)
- [Everything You Wanted to Know About LLM-based Vulnerability Detection But Were Afraid to Ask](https://mlsanity.com/paper/2504.13474/) (2025)
- [A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency](https://mlsanity.com/paper/2505.01658/) (2025)
- [Don't believe everything you read: Understanding and Measuring MCP Behavior under Misleading Tool Descriptions](https://mlsanity.com/paper/2602.03580/) (2026)
- [DocPrism: Multi-lingual Detection of Incorrectness Inconsistencies between Code and Documentation](https://mlsanity.com/paper/2511.00215/) (2025)
- [You Can REST Now: Automated REST API Documentation and Testing via LLM-Assisted Request Mutations](https://mlsanity.com/paper/2402.05102/) (2024)

## Built on by (most cited first, 21 of 21)

- [Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering](https://mlsanity.com/paper/2604.08224/) (2026)
- [SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills](https://mlsanity.com/paper/2604.06550/) (2026)
- [Benign in Isolation, Harmful in Composition: Security Risks in Agent Skill Ecosystems](https://mlsanity.com/paper/2606.15242/) (2026)
- [HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?](https://mlsanity.com/paper/2604.15415/) (2026)
- [Skills on the Fly: Test-Time Adaptive Skill Synthesis for LLM Agents](https://mlsanity.com/paper/2605.16986/) (2026)
- [VIGIL: Runtime Enforcement of Behavioral Specifications in AI Agent Skills](https://mlsanity.com/paper/2606.26524/) (2026)
- [From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills](https://mlsanity.com/paper/2607.01456/) (2026)
- [Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems](https://mlsanity.com/paper/2609.00006/) (2026)
- [MalSkills: Detecting Malicious Skills in the Agentic Supply Chain via Neuro-symbolic Reasoning](https://mlsanity.com/paper/2603.27204/) (2026)
- [SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent System](https://mlsanity.com/paper/2605.09341/) (2026)
- [Trust Me, Import This: Dependency Steering Attacks via Malicious Agent Skills](https://mlsanity.com/paper/2605.09594/) (2026)
- [No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skills](https://mlsanity.com/paper/2605.13044/) (2026)
- [Seeing Is Not Screening: Multimodal Hidden Instruction Attacks on Agent Skill Scanners](https://mlsanity.com/paper/2606.18198/) (2026)
- [Agent Skill Security: Threat Models, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2607.13987/) (2026)
- [Responsible Agentic AI Requires Explicit Provenance](https://mlsanity.com/paper/2605.17169/) (2026)
- [Position: Academic Conferences are Potentially Facing Denominator Gaming Caused by Fully Automated Scientific Agents](https://mlsanity.com/paper/2605.09915/) (2026)
- [Red-Teaming Agent Execution Contexts: Open-World Security Evaluation on OpenClaw](https://mlsanity.com/paper/2605.11047/) (2026)
- [Proteus: A Self-Evolving Red Team for Agent Skill Ecosystems](https://mlsanity.com/paper/2605.11891/) (2026)
- [SkillFuzz: Fuzzing Skill Composition for Implicit Intents Discovery in Open Skill Marketplaces](https://mlsanity.com/paper/2607.02345/) (2026)
- [Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Security](https://mlsanity.com/paper/2608.29596/) (2026)
- [Stealth Apart, Harm Together: Skill Cascading Attacks on Skill-Based Agent Systems](https://mlsanity.com/paper/2609.30383/) (2026)

## Abstract (arXiv)

With the rapid evolution of Large Language Model (LLM) agent ecosystems, centralized skill marketplaces have emerged as pivotal infrastructure for augmenting agent capabilities. However, these marketplaces face unprecedented security challenges, primarily stemming from semantic-behavioral inconsistency and inter-skill combinatorial risks, where individually benign skills induce malicious behaviors during collaborative invocation. To address these vulnerabilities, we propose SkillProbe, a multi-stage security auditing framework driven by multi-agent collaboration. SkillProbe introduces a "Skills-for-Skills" design paradigm, encapsulating auditing processes into standardized skill modules to drive specialized agents through a rigorous pipeline, including admission filtering, semantic-behavioral alignment detection, and combinatorial risk simulation. We conducted a large-scale evaluation using 8 mainstream LLM series across 2,500 real-world skills from ClawHub. Our results reveal a striking popularity-security paradox, where download volume is not a reliable proxy for security quality, as over 90% of high-popularity skills failed to pass rigorous auditing. Crucially, we discovered that high-risk skills form a single giant connected component within the risk-link dimension, demonstrating that cascaded risks are systemic rather than isolated occurrences. We hope that SkillProbe will inspire researchers to provide a scalable governance infrastructure for constructing a trustworthy Agentic Web. SkillProbe is accessible for public experience at skillhub.holosai.io.