mlsanityData to 14 Oct 2026

RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources

arXiv 2606.29538 ↗First posted 28 Jun 2026cs.SE cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Skills are a useful abstraction for software agents, turning human and agent experience into reusable procedural knowledge. Yet existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving tutorial videos and other multimodal human resources largely underused. We present RESOURCE2SKILL, a framework that distills multimodal resources, including tutorial videos, repositories, articles, and reference artifacts, into executable skills for software agents. RESOURCE2SKILL organizes these skills as a hierarchical multimodal Skill Wiki, where each entry combines structured text, code, visual examples, metadata, and provenance. This design preserves complementary signals from different resources: videos capture temporal operations and visual effects, code captures executable tool patterns, and articles or artifacts provide conceptual and stylistic grounding. At inference time, agents retrieve and compose relevant skills from the wiki; when coverage is insufficient, the same construction operator can acquire new skills online. Across seven practical authoring domains, RESOURCE2SKILL improves average overall score by +11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells. Ablations confirm the value of multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 37 of its 40 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

37 of 40 references.

References · 37

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  7. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  11. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
25 more
  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  3. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  5. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  6. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  7. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  8. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  9. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  10. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  11. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  12. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,210
  13. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  14. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  15. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  16. 2022PAL: Program-aided Language Modelscited by 652
  17. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  18. 2024Agent Workflow Memorycited by 383
  19. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
  20. 2019Cross-task weakly supervised learning from instructional videoscited by 199
  21. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 156
  22. 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 113
  23. 2025Inducing Programmatic Skills for Agentic Taskscited by 111
  24. 2026SKILLFOUNDRY: Building Self-Evolving Agent Skill Libraries from Heterogeneous Scientific Resourcescited by 41
  25. 2026SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agentscited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.