# AI Copilots for Reproducibility in Science: A Case Study

arXiv: https://arxiv.org/abs/2506.20130 · HTML: https://mlsanity.com/paper/2506.20130/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 21 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 1 of 1)

- [Sources of Irreproducibility in Machine Learning: A Review](https://mlsanity.com/paper/2204.07610/) (2022)

## Built on by (most cited first, 2 of 2)

- [Enhancing Automated Paper Reproduction via Prompt-Free Collaborative Agents](https://mlsanity.com/paper/2512.02812/) (2025)
- [The Shift Toward Open and Reproducible AI Research](https://mlsanity.com/paper/2606.16974/) (2026)

## Abstract (arXiv)

Open science initiatives seek to make research outputs more transparent, accessible, and reusable, but ensuring that published findings can be independently reproduced remains a persistent challenge. In this paper we describe an AI-driven "Reproducibility Copilot" that analyzes manuscripts, code, and supplementary materials to generate structured Jupyter Notebooks and recommendations aimed at facilitating computational, or "rote", reproducibility. Our initial results suggest that the copilot has the potential to substantially reduce reproduction time (in one case from over 30 hours to about 1 hour) while achieving high coverage of figures, tables, and results suitable for computational reproduction. The system systematically detects barriers to reproducibility, including missing values for hyperparameters, undocumented preprocessing steps, and incomplete or inaccessible datasets. Although preliminary, these findings suggest that AI tools can meaningfully reduce the burden of reproducibility efforts and contribute to more transparent and verifiable scientific communication.