mlsanityData to 14 Oct 2026

AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage

arXiv 2505.20662 ↗First posted 27 May 2025cs.AIcited by 32

Not on the map. Why?

Abstract (arXiv)

Efficient reproduction of research papers is pivotal to accelerating scientific progress. However, the increasing complexity of proposed methods often renders reproduction a labor-intensive endeavor, necessitating profound domain expertise. To address this, we introduce the paper lineage, which systematically mines implicit knowledge from the cited literature. This algorithm serves as the backbone of our proposed ours, a multi-agent framework designed to autonomously reproduce experimental code in a complete, end-to-end manner. To ensure code executability, ours incorporates a sampling-based unit testing strategy for rapid validation. To assess reproduction capabilities, we introduce ourbench, a benchmark featuring verified implementations, alongside comprehensive metrics for evaluating both reproduction and execution fidelity. Extensive evaluations on PaperBench and ourbench demonstrate that ours consistently surpasses existing baselines across all metrics. Notably, it yields substantial improvements in reproduction fidelity and final execution performance. The code is available at https://github.com/AI9Stars/AutoReproduce.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 44 of its 48 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

44 of 48 references.

References · 44

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  5. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  6. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  7. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
  8. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
  9. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  10. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  11. 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
  12. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
32 more
  1. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  2. 2022Decoupled Knowledge Distillationcited by 388
  3. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  4. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
  5. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  6. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  7. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  8. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 210
  9. 2022SimVP: Simpler yet Better Video Predictioncited by 184
  10. 2023SciMON: Scientific Inspiration Machines Optimized for Noveltycited by 179
  11. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 148
  12. 2021TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generationcited by 142
  13. 2023Low-Light Image Enhancement with Wavelet-based Diffusion Modelscited by 127
  14. 2023Dual Aggregation Transformer for Image Super-Resolutioncited by 124
  15. 2024CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmarkcited by 123
  16. 2024Chain of Ideas: Revolutionizing Research Via Novel Idea Development with LLM Agentscited by 97
  17. 2023Solving High-Dimensional PDEs with Latent Spectral Modelscited by 89
  18. 2023HumanMAC: Masked Motion Completion for Human Motion Predictioncited by 81
  19. 2023AutoML-GPT: Automatic Machine Learning with GPTcited by 81
  20. 2025AgentRxiv: Towards Collaborative Autonomous Researchcited by 72
  21. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 70
  22. 2023IEBins: Iterative Elastic Bins for Monocular Depth Estimationcited by 54
  23. 2024SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositoriescited by 52
  24. 2021Tree Decomposed Graph Neural Networkcited by 35
  25. 2022Blurring-Sharpening Process Models for Collaborative Filteringcited by 35
  26. 2024LLM-Select: Feature Selection with Large Language Modelscited by 30
  27. 2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Surveycited by 19
  28. 2025ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologiescited by 15
  29. 2024AutoMMLab: Automatically Generating Deployable Models from Language Instructions for Computer Vision Taskscited by 14
  30. 2024AutoM3L: An Automated Multimodal Machine Learning Framework with Large Language Modelscited by 13
  31. 2024ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generationcited by 11
  32. 2023JarviX: A LLM No code Platform for Tabular Data Analysis and Optimizationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 26.

32 papers.

Sort:

By area:

  1. 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 37
  2. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 23
  3. 2026ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Researchcited by 17
  4. 2026AI for Auto-Research: Roadmap & User Guidecited by 11
  5. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 9
  6. 2026Training AI Scientists to Replicate Researchcited by 9
  7. 2026What Papers Don't Tell You: Recovering Tacit Knowledge for Automated Paper Reproductioncited by 6
  8. 2026Can LLMs Write Reliable Rubrics? A Meta-Evaluation for Experiment Reproductioncited by 6
  9. 2026HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Executioncited by 5
  10. 2026Read the Paper, Write the Code: Agentic Reproduction of Social-Science Resultscited by 5
  11. 2026EchoRL: Reinforcement Learning via Rollout Echoingcited by 4
  12. 2026NERFIFY: A Multi-Agent Framework for Turning NeRF Papers into Codecited by 3
20 more
  1. 2026SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences?cited by 3
  2. 2026Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarkingcited by 2
  3. 2026FactReview: Evidence-Grounded Peer Review with Execution-Based Claim Verificationcited by 2
  4. 2026From paper to benchmark: agentic, framework-based reproduction of under-specified methods in machine health intelligencecited by 2
  5. 2026ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issuescited by 2
  6. 2026VERITAS: Towards a General-Purpose Replication Tool for Scientific Researchcited by 2
  7. 2026Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Paperscited by 2
  8. 2025What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representationscited by 1
  9. 2025Automating Hardware Design and Verification from Architectural Papers via a Neural-Symbolic Graph Frameworkcited by 1
  10. 2025Enhancing Automated Paper Reproduction via Prompt-Free Collaborative Agentscited by 1
  11. 2026An Agentic Approach Towards Replication Package Quality Evaluationcited by 1
  12. 2026ARAC: Benchmarking Auto-Research's Alignment and Completeness on End-to-End Researchscited by 1
  13. 2026ReproAgent: Contract-Guided Paper-to-Code Reproductioncited by 1
  14. 2026TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agentscited by 1
  15. 2026LabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI Agentscited by 1
  16. 2025Automated Reproducibility Has a Problem Statement Problemcited by 0
  17. 2026PaperCompiler: Faithful Paper-to-Code Generation via Repository-Level Specification Compilationcited by 0
  18. 2026PaperDoctor: Evidence-Grounded and Actionable Feedback for Scientific Papers in Progresscited by 0
  19. 2026RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?cited by 0
  20. 2026SPLATIFY: Reproduce, Discover, Innovate! From Papers and Ideas to Trainable 3DGS Codecited by 0