ReproAgent: Contract-Guided Paper-to-Code Reproduction
Not on the map. Why?
Abstract (arXiv)
Paper-to-code reproduction asks scientific AI agents to turn research papers into executable repositories that preserve the paper's method, protocol and artifacts. This is difficult because the specification is split: explicit paper content such as algorithms, metrics and artifacts is often lost across long agent trajectories, while implicit details such as framework defaults and conventions inherited from related work are absent from the paper. We introduce ReproAgent, a four-stage Prepare--Plan--Generate--Repair pipeline built around a persistent implementation contract with two channels: an implementation-requirement channel that turns paper snippets into code obligations, and a reference-evidence channel that retrieves content and structure evidence from related repositories. Both are bound to work packages, projected into file-level contracts, and consumed across generation and repair. On PaperBench Code-Dev, ReproAgent reaches the highest mean score among same-backbone scaffolds under both Claude-Sonnet-4.5 and Gemini-3-Flash. End-to-end channel ablations and per-paper cases support the contribution of both channels. Code and experimental artifacts are publicly available.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 25 Aug 2026.
Builds on
21 of 37 references.
References · 21
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 469
- 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
- 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 367
- 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
- 2022DocPrompting: Generating Code by Retrieving the Docscited by 153
- 2024CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmarkcited by 123
- 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 70
- 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Paperscited by 43
- 2024Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completioncited by 40
- 2025AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineagecited by 32
9 more
- 2025LMR-BENCH: Evaluating LLM Agent's Ability on Reproducing Language Modeling Researchcited by 29
- 2025AutoP2C: An LLM-Based Agent Framework for Code Repository Generation from Multimodal Content in Academic Paperscited by 20
- 2025GraphCodeAgent: Dual Graph-Guided LLM Agent for Retrieval-Augmented Repo-Level Code Generationcited by 17
- 2026Toward Autonomous Long-Horizon Engineering for ML Researchcited by 17
- 2024DocCGen: Document-based Controlled Code Generationcited by 11
- 2025Reflective Paper-to-Code Reproduction Enabled by Fine-Grained Verificationcited by 10
- 2026SciCoQA: Quality Assurance for Scientific Paper--Code Alignmentcited by 10
- 2017DLPaper2Code: Auto-generation of Code from Deep Learning Research Paperscited by 8
- 2026HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Executioncited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.