# Enhancing Automated Paper Reproduction via Prompt-Free Collaborative Agents

arXiv: https://arxiv.org/abs/2512.02812 · HTML: https://mlsanity.com/paper/2512.02812/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 2 Dec 2025.

## Builds on (most cited first, 25 of 25)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [A Survey on Large Language Models for Code Generation](https://mlsanity.com/paper/2406.00515/) (2024)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025)
- [AutoAgents: A Framework for Automatic Agent Generation](https://mlsanity.com/paper/2309.17288/) (2023)
- [MapCoder: Multi-Agent Code Generation for Competitive Problem Solving](https://mlsanity.com/paper/2405.11403/) (2024)
- [Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning](https://mlsanity.com/paper/2504.17192/) (2025)
- [CodeScientist: End-to-End Semi-Automated Scientific Discovery with Code-based Experimentation](https://mlsanity.com/paper/2503.22708/) (2025)
- [CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging](https://mlsanity.com/paper/2502.05664/) (2025)
- [AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage](https://mlsanity.com/paper/2505.20662/) (2025)
- [AutoP2C: An LLM-Based Agent Framework for Code Repository Generation from Multimodal Content in Academic Papers](https://mlsanity.com/paper/2504.20115/) (2025)
- [RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation](https://mlsanity.com/paper/2509.16198/) (2025)
- [Reflective Paper-to-Code Reproduction Enabled by Fine-Grained Verification](https://mlsanity.com/paper/2508.16671/) (2025)
- [AdaCoder: An Adaptive Planning and Multi-Agent Framework for Function-Level Code Generation](https://mlsanity.com/paper/2504.04220/) (2025)
- [LockForge: Automating Paper-to-Code for Logic Locking with Multi-Agent Reasoning LLMs](https://mlsanity.com/paper/2511.18531/) (2025)
- [CodeRefine: A Pipeline for Enhancing LLM-Generated Code Implementations of Research Papers](https://mlsanity.com/paper/2408.13366/) (2024)
- [AI Copilots for Reproducibility in Science: A Case Study](https://mlsanity.com/paper/2506.20130/) (2025)

## Built on by (most cited first, 1 of 1)

- [SA-Bench: Evaluating Semantic Alignment in LLM-Based Paper Reproduction](https://mlsanity.com/paper/2608.24252/) (2026)

## Abstract (arXiv)

Automated paper reproduction has emerged as a promising approach to accelerate scientific research, employing multi-step workflow frameworks to systematically convert academic papers into executable code. However, existing frameworks often lack mechanisms to verify and refine the outputs at each generation step, or rely heavily on manually designed prompts for self-refinement, which limits their adaptability and scalability. To address these limitations, we propose a prompt-free collaborative agent framework that automatically enhances the quality of paper-to-code generation. Our approach employs two collaborative agents: a verification agent that examines whether the outputs at each step satisfy the requirements specified in the corresponding system prompt, and a refinement agent that revises the outputs based on the identified issues. Unlike previous methods that require human experts to craft specific refinement prompts for each step, our framework achieves automatic verification and improvement by leveraging only the original system prompts. We integrate our collaborative agents into the Paper2Code framework and conduct comprehensive experiments on PaperBench Code-Dev and Paper2CodeBench datasets. Experimental results demonstrate that our approach significantly improves the accuracy and completeness of reproduced code, achieving performance gains of approximately 15% and 13%, respectively, compared to the baseline without our agents. Furthermore, comparative experiments against Self-Refine validate the robustness and consistency of our prompt-free approach across different datasets.