Long SKILL Compliance as Logical Reasoning: Closure-Grounded Detection with Scaling-Guided On-Policy Distillation
On the map: LLM reasoning → Reasoning for agents, planning and code
Abstract (arXiv)
The increasing complexity of enterprise business scenarios has promoted the widespread adoption of long SKILL documents in agent systems, posing new challenges for compliance detection: large models incur substantial inference costs, while small models may fail to maintain detection accuracy. To address this gap, we propose SkillCDG, a graph-based framework for long SKILL compliance detection. SkillCDG represents complex business policies as a two-layer constraint dependency graph, where the upper layer indexes SKILL descriptions for scenario routing and the lower layer captures dependencies among atomic constraints within each SKILL. During inference, two-level retrieval followed by dependency closure supports compliance judgment and source traceability. We comprehensively evaluate the framework on three enterprise datasets and two controlled public benchmark variants. Experimental results demonstrate that SkillCDG outperforms baseline methods by up to 12.8 percentage points in detection F1 score, while reducing token consumption by a maximum 64.3%. Moreover, we further investigate the inherent relationships among policy-graph complexity, model scale, and detection performance. Comparative experiments conducted on four checkpoints from a single model family validate a concise and effective scaling trend: end-to-end detection correctness exhibits a complexity-differentiated scaling pattern, and the complexity metric derived from the constraint dependency graph can effectively quantify instance difficulty and the performance improvement potential of models. Leveraging this insightful scaling trend, we conduct adaptive training sample selection and adopt on-policy distillation to efficiently enhance the compliance detection capability of small-scale models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 28 of its 39 references that are in the map. First posted on arXiv 8 Aug 2026.
Builds on
28 of 39 references.
References · 28
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
- 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
- 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
- 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 310
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
16 more
- 2025Generalizing Verifiable Instruction Followingcited by 178
- 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
- 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 118
- 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 94
- 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
- 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
- 2025KGGen: Extracting Knowledge Graphs from Plain Text with Language Modelscited by 49
- 2026SkillLearnBench: Benchmarking Continual Learning Methods for Agent Skill Generation on Real-World Taskscited by 45
- 2025AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarioscited by 44
- 2026Skill Retrieval Augmentation for Agentic AIcited by 41
- 2024LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarioscited by 31
- 2026Beyond IVR: Benchmarking Customer Support LLM Agents for Business-Adherencecited by 11
- 2026SkillTester: Benchmarking Utility and Security of Agent Skillscited by 10
- 2026CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systemscited by 5
- 2025Predicting Task Performance with Context-aware Scaling Lawscited by 2
- 2026SkillAudit: From Fixed-Suite Benchmarking to Skill-Centered Assessmentcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SkillConsist: Detecting Inconsistencies in Agent Skills via Bidirectional Graph Alignment
- 2026Skills Are Not Islands: Measuring Dependency and Risk in Agent Skill Supply Chains
- 2026Policy-Invisible Violations in LLM-Based Agents
- 2026AgentLTL: A Trace-Verification Framework for Measuring, Enforcing, and Training Procedural Compliance in Tool-Using LLM Agents
- 2026MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents
- 2026Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning
- 2026Skill Coverage: A Test Adequacy Metric for Agent Skills
- 2026Near-Miss: Latent Policy Failure Detection in Agentic Workflows