mlsanityData to 14 Oct 2026

SkillScreen: High-Recall Screening of Malicious Agent Skills with Decision Models

arXiv 2610.17727 ↗First posted 13 Oct 2026cs.CR cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → Agent skill learning and evolution also fits: Agent threats, guardrails and governance

Abstract (arXiv)

Agent skills turn natural-language instructions into actions, creating a security challenge: legitimate automation and malicious behaviour can look deceptively similar. A practical screen must catch nearly every attack while keeping the review load manageable. We present SkillScreen, a compact decision model trained to score skill maliciousness directly from SKILL.md in a single forward pass. On MaliciousSkillBench, SkillScreen flags 20-25% of benign skills at 99% recall on unseen attack families, substantially fewer than the word-level classifier and prompted LLM judges tested. Under the same training protocol, it also generalizes better than a fully fine-tuned encoder to unseen sources. Yet unfamiliar benign sources sharply increase false positives, showing that recognizing new attacks does not ensure reliable screening of new skill collections. Our results demonstrate the value of task-trained decision models for high-recall skill screening. Project page: https://jerroldh.github.io/SkillScreen-webpage/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 28 of its 45 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

28 of 45 references.

References · 28

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  5. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  6. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  7. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  8. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  9. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  10. 2019Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approachcited by 308
  11. 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
  12. 2021Entailment as Few-Shot Learnercited by 123
16 more
  1. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 90
  2. 2025SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agentscited by 71
  3. 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploitscited by 63
  4. 2020Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attackscited by 61
  5. 2026"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wildcited by 50
  6. 2026Formal Analysis and Supply Chain Security for Agentic AI Skillscited by 25
  7. 2026SkillTrojan: Backdoor Attacks on Skill-Based Agent Systemscited by 25
  8. 2026How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Studycited by 19
  9. 2026SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfacescited by 19
  10. 2026MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skillscited by 19
  11. 2026AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skillscited by 12
  12. 2026AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitorcited by 10
  13. 2022Practical Automated Detection of Malicious npm Packagescited by 8
  14. 2026SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Constructioncited by 6
  15. 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detectioncited by 1
  16. 2026RH-Detect: A Unified Benchmark for Reward Hacking Detectioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills
  2. 2026SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills
  3. 2026SkillSecurer: Detecting and Patching Prompt-Injection Vulnerabilities in AI Agent Skills
  4. 2026SkillAtlas: An Attack Trace Library for Agent Skills
  5. 2026Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents
  6. 2026Detecting Malicious Agent Skills in the Wild using Attention
  7. 2026Towards a Risk Assessment of Malicious Skill Files in Coding Agents
  8. 2026RouteGuard: Internal-Signal Detection of Skill Poisoning in LLM Agents