# AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage

arXiv: https://arxiv.org/abs/2505.20662 · HTML: https://mlsanity.com/paper/2505.20662/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 48 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 30 of 44)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [MM-LLMs: Recent Advances in MultiModal Large Language Models](https://mlsanity.com/paper/2401.13601/) (2024)
- [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024)
- [ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models](https://mlsanity.com/paper/2404.07738/) (2024)
- [SimVP: Simpler yet Better Video Prediction](https://mlsanity.com/paper/2206.05099/) (2022)
- [SciMON: Scientific Inspiration Machines Optimized for Novelty](https://mlsanity.com/paper/2305.14259/) (2023)
- [CycleResearcher: Improving Automated Research via Automated Review](https://mlsanity.com/paper/2411.00816/) (2024)
- [TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generation](https://mlsanity.com/paper/2111.08095/) (2021)
- [Low-Light Image Enhancement with Wavelet-based Diffusion Models](https://mlsanity.com/paper/2306.00306/) (2023)
- [Dual Aggregation Transformer for Image Super-Resolution](https://mlsanity.com/paper/2308.03364/) (2023)
- [CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark](https://mlsanity.com/paper/2409.11363/) (2024)
- [Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agents](https://mlsanity.com/paper/2410.13185/) (2024)
- [Solving High-Dimensional PDEs with Latent Spectral Models](https://mlsanity.com/paper/2301.12664/) (2023)
- [HumanMAC: Masked Motion Completion for Human Motion Prediction](https://mlsanity.com/paper/2302.03665/) (2023)

## Built on by (most cited first, 30 of 32)

- [ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLM](https://mlsanity.com/paper/2506.14766/) (2025)
- [Self-Evolving Multi-Agent Systems via Textual Backpropagation](https://mlsanity.com/paper/2506.09046/) (2025)
- [ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research](https://mlsanity.com/paper/2606.07591/) (2026)
- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Training AI Scientists to Replicate Research](https://mlsanity.com/paper/2608.13331/) (2026)
- [What Papers Don't Tell You: Recovering Tacit Knowledge for Automated Paper Reproduction](https://mlsanity.com/paper/2603.01801/) (2026)
- [Can LLMs Write Reliable Rubrics? A Meta-Evaluation for Experiment Reproduction](https://mlsanity.com/paper/2607.12835/) (2026)
- [HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution](https://mlsanity.com/paper/2604.17745/) (2026)
- [Read the Paper, Write the Code: Agentic Reproduction of Social-Science Results](https://mlsanity.com/paper/2604.21965/) (2026)
- [EchoRL: Reinforcement Learning via Rollout Echoing](https://mlsanity.com/paper/2605.31228/) (2026)
- [NERFIFY: A Multi-Agent Framework for Turning NeRF Papers into Code](https://mlsanity.com/paper/2603.00805/) (2026)
- [SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences?](https://mlsanity.com/paper/2604.10718/) (2026)
- [Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarking](https://mlsanity.com/paper/2602.24009/) (2026)
- [FactReview: Evidence-Grounded Peer Review with Execution-Based Claim Verification](https://mlsanity.com/paper/2604.04074/) (2026)
- [From paper to benchmark: agentic, framework-based reproduction of under-specified methods in machine health intelligence](https://mlsanity.com/paper/2605.28371/) (2026)
- [ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues](https://mlsanity.com/paper/2606.18237/) (2026)
- [VERITAS: Towards a General-Purpose Replication Tool for Scientific Research](https://mlsanity.com/paper/2607.02931/) (2026)
- [Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Papers](https://mlsanity.com/paper/2609.11117/) (2026)
- [What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representations](https://mlsanity.com/paper/2510.17795/) (2025)
- [Automating Hardware Design and Verification from Architectural Papers via a Neural-Symbolic Graph Framework](https://mlsanity.com/paper/2511.06067/) (2025)
- [Enhancing Automated Paper Reproduction via Prompt-Free Collaborative Agents](https://mlsanity.com/paper/2512.02812/) (2025)
- [An Agentic Approach Towards Replication Package Quality Evaluation](https://mlsanity.com/paper/2606.02006/) (2026)
- [ARAC: Benchmarking Auto-Research's Alignment and Completeness on End-to-End Researchs](https://mlsanity.com/paper/2608.12788/) (2026)
- [ReproAgent: Contract-Guided Paper-to-Code Reproduction](https://mlsanity.com/paper/2608.24291/) (2026)
- [TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents](https://mlsanity.com/paper/2609.05079/) (2026)
- [LabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI Agents](https://mlsanity.com/paper/2609.13437/) (2026)
- [Automated Reproducibility Has a Problem Statement Problem](https://mlsanity.com/paper/2601.04226/) (2025)
- [PaperCompiler: Faithful Paper-to-Code Generation via Repository-Level Specification Compilation](https://mlsanity.com/paper/2609.02272/) (2026)
- [PaperDoctor: Evidence-Grounded and Actionable Feedback for Scientific Papers in Progress](https://mlsanity.com/paper/2609.16995/) (2026)

## Abstract (arXiv)

Efficient reproduction of research papers is pivotal to accelerating scientific progress. However, the increasing complexity of proposed methods often renders reproduction a labor-intensive endeavor, necessitating profound domain expertise. To address this, we introduce the paper lineage, which systematically mines implicit knowledge from the cited literature. This algorithm serves as the backbone of our proposed ours, a multi-agent framework designed to autonomously reproduce experimental code in a complete, end-to-end manner. To ensure code executability, ours incorporates a sampling-based unit testing strategy for rapid validation. To assess reproduction capabilities, we introduce ourbench, a benchmark featuring verified implementations, alongside comprehensive metrics for evaluating both reproduction and execution fidelity. Extensive evaluations on PaperBench and ourbench demonstrate that ours consistently surpasses existing baselines across all metrics. Notably, it yields substantial improvements in reproduction fidelity and final execution performance. The code is available at https://github.com/AI9Stars/AutoReproduce.