ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation
On the map: LLM agents and tool use → Agent skill learning and evolution
Abstract (arXiv)
Skill-distillation pipelines learn reusable rules from LLM agent trajectories, but they lack a key signal: how much each step costs. Without per-step cost, a pipeline cannot distinguish adding a missing step to fix a bug from removing an expensive step that never affected the outcome. We use the cost-attribution gap to ask whether the rule types inside a distilled skill transfer the same way to new tasks. ClawTrace records cost-attributed agent traces and compiles each session into a TraceCard; CostCraft reads TraceCards and writes three kinds of skill patches: preserve, prune, and repair. We find a pattern aggregate metrics hide. On 30 held-out SpreadsheetBench tasks across two seeds, removing prune patches roughly tripled the quality-regression count without lowering median cost. Across the full 84-task SkillsBench transfer, CostCraft saves no aggregate cost. All three quality regressions trace to the preserve lane, and both quality wins trace to the prune lane: prune patches act as quality guardrails while preserve patches drive regressions. We argue that reusable agent skills should be evaluated at the rule-type level, not as monolithic instruction packages. To support this, we release ClawTrace, the TraceCard schema, and the full set of typed skills.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use. Builds on 20 of its 30 references that are in the map. First posted on arXiv 26 Apr 2026.
Builds on
20 of 30 references.
Closest ancestors · 6
Same direction, newest first
- 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification2 Apr
- 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills26 Mar
- 2026EvoSkill: Automated Skill Discovery for Multi-Agent Systems3 Mar
- 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks13 Feb
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward12 Feb
- 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning9 Feb
Other references · 14
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
- 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
- 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 174
- 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolutioncited by 105
- 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 100
- 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
- 2024AgentOps: Enabling Observability of LLM Agentscited by 46
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Cost-Utility Alignment in LLM Agent Trajectories:Profiling,Attribution,Diagnosis,Adaptation,and Evaluationcited by 1
- 2026ParEvalLayer: When Partial LLM-Agent Evaluations Support a Decisioncited by 0
- 2026AgentBetta: Verification-Driven Adaptive Configuration of an AI Nano-Agent through Selective Expansion and Verified Contractioncited by 0
- 2026Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnessescited by 0
- 2026Distilling Agentic Systems: A Roadmap across Models, Artifacts, and Harnesses29 Sep 2026
- 2026AgentBetta: Verification-Driven Adaptive Configuration of an AI Nano-Agent through Selective Expansion and Verified Contraction20 Sep 2026
- 2026Cost-Utility Alignment in LLM Agent Trajectories:Profiling,Attribution,Diagnosis,Adaptation,and Evaluation25 Aug 2026
- 2026ParEvalLayer: When Partial LLM-Agent Evaluations Support a Decision3 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026What Should a Skill Remember? Quality--Cost Trade-offs in Cost-Aware Skill Rewriting for Language Model Agents
- 2026SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse
- 2026SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revision
- 2026SkillRouter: Skill Routing for LLM Agents at Scale
- 2026SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use
- 2026Counterfactual Trace Auditing of LLM Agent Skills
- 2026SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing
- 2026SkillZip: Evaluation-Free Skill Compression for Self-Evolving Agents by Discovering Reusable Structure