Evidence-Calibrated Runtime Reconstruction for Agent Skills Across Heterogeneous Coding Agents
On the map: LLM agents and tool use → Agent skill learning and evolution
Abstract (arXiv)
Agent Skills package reusable instructions and assets for tool-using language-model agents. Progressive loading creates failure boundaries poorly represented by session-, model-, or tool-centric traces: a Skill can be discovered but not activated, activated without instructions, or appear successful without an independently verified outcome. We present Skill Runtime Intelligence, a passive runtime-intelligence system that reconstructs supported Skill-lifecycle stages across heterogeneous harnesses while preserving unsupported stages as unknown. Its Run Panorama separates immutable events, deterministic relations, inferred diagnoses, and controlled outcomes with four evidence grades; optional trace import and OTLP/HTTP export support existing observability deployments. Across six frozen repository profiles, three coding agents, and seven clean or fault-injected conditions, all 126 executions preserve source worktrees and each correlates to exactly one source session. Yet adapters expose three distinct semantics: no Skill runs; complete runs but no failure-like events; or failure-like events in every operational-failure and clean session. In a seven-template diagnostic study, semantic aliases and Panorama localize the same six non-clean boundaries but differ in exact/status behavior; both Raw views emit a failure status on all 18 clean cases, while Panorama emits none. A known-rule graph conforms to 126/126 frozen contracts, whereas a second model completes only 228/378 calls. These observations motivate executable adapter qualification and show that event presence is not boundary fidelity, composite exact scores mask distinct errors, and model explanations must not overwrite deterministic facts.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 13 of its 24 references that are in the map. First posted on arXiv 9 Aug 2026.
Builds on
13 of 24 references.
Closest ancestors · 2
Same direction, newest first
- 2026Skill Coverage: A Test Adequacy Metric for Agent Skills9 Jun
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks13 Feb
Other references · 11
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
- 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
- 2026AgentRx: Diagnosing AI Agent Failures from Execution Trajectoriescited by 53
- 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflowscited by 44
- 2026Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systemscited by 22
- 2026From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flawscited by 21
- 2025On the Use of Agentic Coding Manifests: An Empirical Study of Claude Codecited by 14
- 2026Auditing Agent Harness Safetycited by 8
- 2026AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agentscited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior
- 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries
- 2026SKILLFOUNDRY: Building Self-Evolving Agent Skill Libraries from Heterogeneous Scientific Resources
- 2026SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revision
- 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications
- 2026SkillZip: Evaluation-Free Skill Compression for Self-Evolving Agents by Discovering Reusable Structure
- 2026Subagents vs Agent Skills: Executing Reusable Knowledge for Long-Horizon Agentic Tasks
- 2026SkillHone: A Harness for Continual Agent Skill Evolution Through Persistent Decision History