# SkillConsist: Detecting Inconsistencies in Agent Skills via Bidirectional Graph Alignment

arXiv: https://arxiv.org/abs/2608.07639 · HTML: https://mlsanity.com/paper/2608.07639/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Placed in

- LLM agents and tool use / [Agent skill learning and evolution](https://mlsanity.com/agents/agent-skill-learning-and-evolution/)

## Builds on (most cited first, 18 of 18)

- [AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents](https://mlsanity.com/paper/2503.18666/) (2025)
- [Prompt Injection Attack to Tool Selection in LLM Agents](https://mlsanity.com/paper/2504.19793/) (2025)
- [Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale](https://mlsanity.com/paper/2601.10338/) (2026)
- [Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks](https://mlsanity.com/paper/2602.20156/) (2026)
- [ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback](https://mlsanity.com/paper/2601.10156/) (2026)
- [SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents](https://mlsanity.com/paper/2602.14211/) (2026)
- [Deep Just-In-Time Inconsistency Detection Between Comments and Source Code](https://mlsanity.com/paper/2010.01625/) (2020)
- [SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills](https://mlsanity.com/paper/2604.06550/) (2026)
- [MalSkillBench: A Runtime-Verified Benchmark of Malicious Agent Skills](https://mlsanity.com/paper/2606.07131/) (2026)
- [Skill Coverage: A Test Adequacy Metric for Agent Skills](https://mlsanity.com/paper/2606.20659/) (2026)
- [Behavioral Integrity Verification for AI Agent Skills](https://mlsanity.com/paper/2605.11770/) (2026)
- [Code Comment Inconsistency Detection with BERT and Longformer](https://mlsanity.com/paper/2207.14444/) (2022)
- [DocPrism: Multi-lingual Detection of Incorrectness Inconsistencies between Code and Documentation](https://mlsanity.com/paper/2511.00215/) (2025)
- [CASCADE: Detecting Inconsistencies between Code and Documentation with Automatic Test Generation](https://mlsanity.com/paper/2604.19400/) (2026)
- [Do Skill Descriptions Tell the Truth? Detecting Undisclosed Security Behaviors in Code-Backed LLM Skills](https://mlsanity.com/paper/2605.12875/) (2026)
- [AIP: A Graph Representation for Learning and Governing Agent Skills](https://mlsanity.com/paper/2606.04781/) (2026)
- [Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications](https://mlsanity.com/paper/2606.04769/) (2026)
- [NEXUS: Structured Runtime Safety for Tool-Using LLM Agents](https://mlsanity.com/paper/2607.19356/) (2026)

## Built on by (most cited first, 1 of 1)

- [Why Static Skill Inconsistencies Do Not Equate to Runtime Risks: A Behavioral Analysis of LLM Agent Execution](https://mlsanity.com/paper/2610.18927/) (2026)

## Abstract (arXiv)

Agent Skills provide reusable capabilities to LLM agents. Agent Skill inconsistencies can expose undisclosed dangerous behavior or cause wrong Skill selection. Recent Agent Skill research has increasingly examined Agent Skill consistency detection. Existing methods evaluate behaviors or security-property graphs against predefined categories or declared scopes. More recently, PL-HCL uses an LLM-based model to learn consistency across metadata, instructions, and resources. However, declaration and implementation behavior can be mixed across text and code, and a concise declaration can correspond to multiple connected implementation steps. We present SkillConsist to address both challenges. An LLM separates declaration and implementation content into behavior records on the implementation and declaration sides, while static analysis supplements implementation records. These records form declaration and implementation behavior graphs, respectively. Starting from a behavior record on either side, bidirectional graph alignment searches the other graph for a candidate subgraph and expands it along behavior relations until it completely expresses the source-side behavior. Graph differencing identifies conflicts between aligned subgraphs and outputs the detection results. We construct a 633-Skill benchmark from ClawHub's 500 most-downloaded public Skills and 133 Skill-Inject packages. The benchmark contains 319 inconsistent and 314 consistent Skills and 442 localized inconsistency annotations. On this benchmark, SkillConsist achieves 86.85% precision, 89.03% recall, and 87.93% F1 for package-level detection, improving F1 over the best baseline by 20.43 percentage points. For localization, it achieves 67.60% precision, 58.14% recall, and 62.52% F1.