# Evidence Over Plans: Online Trajectory Verification for Skill Distillation

arXiv: https://arxiv.org/abs/2605.09192 · HTML: https://mlsanity.com/paper/2605.09192/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 30 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 29 of 29)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks](https://mlsanity.com/paper/2602.12670/) (2026)
- [Large Language Models as Tool Makers](https://mlsanity.com/paper/2305.17126/) (2023)
- [SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning](https://mlsanity.com/paper/2602.08234/) (2026)
- [Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents](https://mlsanity.com/paper/2403.02502/) (2024)
- [Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills](https://mlsanity.com/paper/2603.25158/) (2026)
- [AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials](https://mlsanity.com/paper/2412.09605/) (2024)
- [AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolution](https://mlsanity.com/paper/2603.01145/) (2026)
- [EvoSkill: Automated Skill Discovery for Multi-Agent Systems](https://mlsanity.com/paper/2603.02766/) (2026)
- [SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization](https://mlsanity.com/paper/2604.02268/) (2026)
- [CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models](https://mlsanity.com/paper/2305.14318/) (2023)
- [SkillX: Automatically Constructing Skill Knowledge Bases for Agents](https://mlsanity.com/paper/2604.04804/) (2026)
- [How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings](https://mlsanity.com/paper/2604.04323/) (2026)
- [SkillRouter: Skill Routing for LLM Agents at Scale](https://mlsanity.com/paper/2603.22455/) (2026)
- [SkillNet: Create, Evaluate, and Connect AI Skills](https://mlsanity.com/paper/2603.04448/) (2026)
- [Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate](https://mlsanity.com/paper/2402.02769/) (2024)
- [CUA-Skill: Develop Skills for Computer Using Agent](https://mlsanity.com/paper/2601.21123/) (2026)
- [SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?](https://mlsanity.com/paper/2603.00718/) (2026)
- [Online Experiential Learning for Language Models](https://mlsanity.com/paper/2603.16856/) (2026)
- [M^3-Bench: Multi-Modal, Multi-Hop, Multi-Threaded Tool-Using MLLM Agent Benchmark](https://mlsanity.com/paper/2511.17729/) (2025)
- [LED: LLM Enhanced Open-Vocabulary Object Detection without Human Curated Data Generation](https://mlsanity.com/paper/2503.13794/) (2025)

## Built on by (most cited first, 1 of 1)

- [Agent Skill Evaluation and Evolution: Frameworks and Benchmarks](https://mlsanity.com/paper/2606.11435/) (2026)

## Abstract (arXiv)

Agent skills can remarkably improve task success rates by using human-written procedural documents, but their quality is difficult to assess without environment-grounded verification. Existing skill generation methods heavily rely on preference logs rather than direct environment interaction, often yielding negligible or even degraded gains. We identify that it is a fundamental timing bottleneck: robust skills should be posterior-based, distilled from empirical environment interaction rather than prior plans. In this study, we introduce the Posterior Distillation Index (PDI), a trajectory-level metric that quantifies how well a distilled skill is grounded in the task-environment evidence. To operationalize PDI, we present SPARK (Structured Pipelines for Autonomous Runnable tasKs and sKill generation) for preserving task execution evidence towards full trajectory-level analysis. SPARK generates environment-verified trajectories used to compute PDI, and it applies PDI as an online diagnostic and intervention signal to ensure posterior skill formation. Across 86 runnable tasks, SPARK-generated skills consistently surpass no-skill baselines and outperform human-written skills on student models (inference cost up to 1,000x cheaper than teacher models). These findings show that PDI-guided distillation produces efficient and transferable skills grounded in the task-environment interaction. We release our code at https://github.com/EtaYang10th/spark-skills .