SkillGate: Cost Efficient Runtime Malicious Skill File Detection in Coding Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Software engineering teams now deploy AI coding agents (Cursor, Claude Code, GitHub Copilot) as first-class productivity tools, installing domain-specific skill files to tailor agent behavior to project APIs, framework conventions, and organizational workflows. These complex Markdown files are easily downloaded from public registries with a single npx skills add command and no real security screening, representing a novel supply-chain attack surface: a malicious skill file can silently reprogram agent behavior, exfiltrating credentials, injecting backdoors into generated code, or redirecting agent actions to attacker-controlled endpoints. The threat is not hypothetical: recent reports document hundreds of malicious skill packages in public registries, including organized campaigns that distributed credential-stealing infostealers via fake productivity skills. No systematic toolchain defense exists for this attack surface. We present SkillGate, a deployable security gateway that screens AI skill packages before coding agent installation. SkillGate uses a hybrid regex-prefilter + LLM-judge pipeline: safe-signal files bypass the LLM entirely (skip savings); flagged files have only their matched snippet windows sent to the judge, not the full content (snippet savings). We answer four research questions covering detection effectiveness, screening cost, runtime overhead, and false positive behavior on the SkillsBench benchmark against two existing tools. On SkillsBench (n=1,650, 9.1% malicious), SkillGate achieves F1=0.817, FPR=1.13% while reducing LLM input tokens by 77% vs. full-file screening, and outperforming existing tools by 5-6x on threshold-independent AUPRC (0.830 vs. 0.144/0.162).
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 22 of its 37 references that are in the map. First posted on arXiv 28 Jul 2026.
Builds on
22 of 37 references.
Closest ancestors · 1
Same direction, newest first
Other references · 21
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
9 more
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
- 2022Do Users Write More Insecure Code with AI Assistants?cited by 186
- 2020Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attackscited by 57
- 2022Taxonomy of Attacks on Open-Source Software Supply Chainscited by 57
- 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
- 2019Small World with High Risks: A Study of Security Threats in the npm Ecosystemcited by 36
- 2022Practical Automated Detection of Malicious npm Packagescited by 7
- 2026Usage, Effects and Requirements for AI Coding Assistants in the Enterprise: An Empirical Studycited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Towards a Risk Assessment of Malicious Skill Files in Coding Agents
- 2026SkillShield: Prompt-Space Security Skills for LLM Coding Agents
- 2026"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild
- 2026PhantomSkill: Malicious Code Injection in Agent Skill Ecosystems
- 2026SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills
- 2026SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills
- 2026MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills
- 2026SkillBloat: Token Amplification Attacks via Skill Injection in LLM Coding Agents