ClawHub Security Signals: When VirusTotal, Static Analysis, and SkillSpector Disagree
Not on the map. Why?
Abstract (arXiv)
Agent skills extend AI agents with reusable instructions, tools, scripts, references, and workflows, establishing a security boundary distinct from both model safety and traditional package-malware detection. ClawHub Security Signals is a sanitized dataset of 67,453 latest public OpenClaw skill versions. Each row pairs redacted SKILL.md content and sanitized bundled files where present with a final ClawScan registry verdict and evidence from three scanner families: VirusTotal, static heuristic analysis, and NVIDIA SkillSpector. Rather than estimating malicious-skill prevalence, we study scanner disagreement. The three scanners rarely flag the same skills: any pair overlaps on at most 10.4% of their combined positives, only 0.69% of skills are flagged by all three, and 81.9% of flagged skills are identified by a single scanner. The disagreement is structured by attack surface. SkillSpector, which raises semantic agentic-risk advisories rather than malware-reputation signals, is positive for 19,209 of 25,504 suspicious rows (75.3%) but only 14 of 206 malicious rows (6.8%). The malicious-verdict region shows the inverse profile: 150 of 206 malicious rows (72.8%) are VirusTotal-positive, consistent with bundled-code malware evidence. These results show that agent-skill security requires layered governance, not single-scanner allow/block decisions. The corpus is released as a sanitized silver-standard dataset: labels are the registry's automated verdicts, not human-annotated ground truth, and the release represents an early, versioned snapshot intended to support the community while a human-annotated subset is developed. Further research is encouraged, including models tailored for skill-security triage.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 12 of its 46 references that are in the map. First posted on arXiv 31 May 2026.
Builds on
12 of 46 references.
References · 12
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
- 2024LLM Agents can Autonomously Exploit One-day Vulnerabilitiescited by 125
- 2024LLM Agents can Autonomously Hack Websitescited by 108
- 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploitscited by 61
- 2020Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attackscited by 57
- 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
- 2019Small World with High Risks: A Study of Security Threats in the npm Ecosystemcited by 36
- 2023An Empirical Study of Malicious Code In PyPI Ecosystemcited by 18
- 2025Mind the Metrics: Patterns for Telemetry-Aware In-IDE AI Application Development using the Model Context Protocol (MCP)cited by 4
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Understanding and mitigating the risks of OpenClaw for non-technical users: A practical guide with Skillcited by 0
- 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detectioncited by 0
- 2026After the Party: Growth, Governance, and Security Scanning in the OpenClaw Agent Skill Ecosystemcited by 0
- 2026Can Agents Trust Their Skills? Uncovering Unsafe Chains of Trust in Skill-Based LLM Agentscited by 0
- 2026Can Agents Trust Their Skills? Uncovering Unsafe Chains of Trust in Skill-Based LLM Agents30 Sep 2026
- 2026After the Party: Growth, Governance, and Security Scanning in the OpenClaw Agent Skill Ecosystem15 Sep 2026
- 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection20 Aug 2026
- 2026Understanding and mitigating the risks of OpenClaw for non-technical users: A practical guide with Skill9 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026SkillGate: Cost Efficient Runtime Malicious Skill File Detection in Coding Agents
- 2026SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills
- 2026Benchmarking Security Risk Detection and Verification in Open Agentic Skill Ecosystems
- 2026MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills
- 2026SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills
- 2026SkillVetBench: LLM-as-Judge for Multi-Dimensional Security Risk Evaluation in Open-Source LLM Agent Skills
- 2026SkillSecurer: Detecting and Patching Prompt-Injection Vulnerabilities in AI Agent Skills
- 2026Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents