mlsanityData to 7 Oct 2026

Demystifying Agent Skills for Smart Contract Auditing: Design, Effectiveness, Behavioral Impact

arXiv 2609.29454 ↗First posted 24 Sep 2026cs.SEcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance also fits: Software engineering coding agents

Abstract (arXiv)

LLM agents, notably Claude Code and OpenAI Codex, are emerging as versatile tools beyond coding agents only. These agents can be enhanced with skills---reusable artifacts that package domain knowledge, workflows, and tool-use instructions. To date, however, little is known about how such skills are designed or how they affect agent effectiveness and behavior in practice. In this paper, we investigate these questions in smart contract security auditing, a domain in which agents have shown substantial promise. We systematically collect 83 smart contract audit skills from the wild and evaluate them on EVMBench across seven agent--model configurations. Our study examines three dimensions: (i) the design characteristics of audit skills, including their structure, knowledge representations, workflows, and tool dependencies; (ii) their effectiveness in improving vulnerability detection; and (iii) their influence on agent execution trajectories. We find that audit skills are mostly lightweight but heterogeneous in design, covering a broad yet imbalanced range of vulnerability types. Their effectiveness is determined primarily by the model rather than the agent harness: Codex/GPT-5.5 achieves the largest gains, improving detection score by 22.8% and captured award by 43.2%. We further find that skill triggering is a key bottleneck. When triggered, skills preserve a shared six-stage audit workflow while exhibiting distinct loading patterns and differential effects on agent behavior across configurations. We release our skill corpus and artifacts to support future research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 18 of its 51 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

18 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale15 Jan
  2. 2024Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justifications24 Mar

Other references · 16

  1. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  2. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 122
  3. 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
  4. 2023GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysiscited by 65
  5. 2026SkillNet: Create, Evaluate, and Connect AI Skillscited by 48
  6. 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projectscited by 45
  7. 2024PropertyGPT: LLM-driven Formal Verification of Smart Contracts through Retrieval-Augmented Property Generationcited by 36
  8. 2026SkillReducer: Optimizing LLM Agent Skills for Token Efficiencycited by 29
  9. 2026EVMbench: Evaluating AI Agents on Smart Contract Securitycited by 7
  10. 2026Re-Evaluating EVMBench: Are AI Agents Ready for Smart Contract Security?cited by 3
  11. 2026SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent Systemcited by 3
  12. 2026Skill Availability and Presentation Granularity in Large-Language-Model Agents: A Controlled SkillsBench Studycited by 3
4 more
  1. 2026Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Composecited by 3
  2. 2025LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recoverycited by 2
  3. 2026OpenClaw-Skill: Collective Skill Tree Search for Agentic Large Language Modelscited by 2
  4. 2026Automating SKILL.md Generation for Computer-Using Agents via Interaction Trajectory Miningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LISA Technical Report: An Agentic Framework for Smart Contract Auditing
  2. 2026Transferable Self-Evolving Playbooks for Agentic Security Auditing
  3. 2026SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaboration
  4. 2026An Effective and Cost-Efficient Agentic Framework for Ethereum Smart Contract Auditing
  5. 2026Beyond Detection: Agentic Attack Synthesis and Simulation for Smart Contracts
  6. 2026Towards Secure Agent Skills: Architecture, Threat Taxonomy, and Security Analysis
  7. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agents
  8. 2026Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization