Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills
Not on the map. Why?
Abstract (arXiv)
Autonomous agents are beginning to carry out machine-learning (ML) research end to end. These agents combine a model backbone with a harness for planning, execution, memory, and verification, but this architecture still leaves domain-specific know-how outside the agent. We call this missing layer operational knowledge, the know-how that separates knowing a method from making it work. That knowledge is not absent from the field. It appears in repositories and papers, but in forms written for human readers and too large to load during a task. Once distilled into compact, verified skills, this knowledge can be reused across tasks rather than rediscovered during each run. We present DisCo, a skill-powered research agent that creates skills and uses them during research. Its distillation runs in two complementary forms: task-agnostic, condensing the field's widely used repositories into reusable skills, and task-oriented, producing the skills a concrete task calls for. The former, applied across the open ecosystem, yields the AREX-Skill Library, with 5,000+ verified skills distilled from 1,000 widely used ML repositories and organized into 20 areas and 178 capability families. With the GPT-5.5 backbone, research harness, and downstream execution budget held fixed, the skill-equipped research agent scores 134.3% higher on MLE-bench, 34.4% higher on PaperBench, 9.2% higher on FrontierCS, and 14.0% higher on PassNet than the same agent without skills. These gains come from adding distilled operating context under that fixed setup.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
32 of 40 references.
References · 32
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
- 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
- 2025The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Searchcited by 375
- 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
- 2024Agent Workflow Memorycited by 342
- 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
- 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
- 2024Data Interpreter: An LLM Agent For Data Sciencecited by 222
20 more
- 2025AIDE: AI-Driven Exploration in the Space of Codecited by 206
- 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 200
- 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
- 2024DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoningcited by 119
- 2025AI-Researcher: Autonomous Scientific Innovationcited by 88
- 2024AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitionscited by 70
- 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
- 2025ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineeringcited by 26
- 2024SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learningcited by 23
- 2026MARS: Modular Agent with Reflective Search for Automated AI Researchcited by 22
- 2026BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing?cited by 20
- 2026MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discoverycited by 20
- 2025FrontierCS: Evolving Challenges for Evolving Intelligencecited by 19
- 2026Toward Autonomous Long-Horizon Engineering for ML Researchcited by 15
- 2025DeepCode: Open Agentic Codingcited by 14
- 2026AIBuildAI: An AI Agent for Automatically Building AI Modelscited by 10
- 2025Reflective Paper-to-Code Reproduction Enabled by Fine-Grained Verificationcited by 9
- 2026From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skillscited by 5
- 2026Reasoning as Gradient: Scaling MLE Agents Beyond Tree Searchcited by 4
- 2026PassNet: Scaling Large Language Models for Graph Compiler Pass Generationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026SkillCenter: A Large-Scale Source-Grounded Skill Library for Autonomous AI Agents
- 2026SKILLFOUNDRY: Building Self-Evolving Agent Skill Libraries from Heterogeneous Scientific Resources
- 2026GitSkills: A Dataset of Agent Skills on GitHub
- 2025SkillFlow: Scalable and Efficient Agent Skill Retrieval System
- 2026From Registry to Repository: How AI Agent Skills Are Written, Adapted, and Maintained
- 2026Subagents vs Agent Skills: Executing Reusable Knowledge for Long-Horizon Agentic Tasks
- 2026SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents
- 2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution