# Automated Modernization of Machine Learning Engineering Notebooks for Reproducibility

arXiv: https://arxiv.org/abs/2602.07195 · HTML: https://mlsanity.com/paper/2602.07195/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 6 Feb 2026.

## Builds on (most cited first, 10 of 10)

- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [Natural Language to Code Generation in Interactive Data Science Notebooks](https://mlsanity.com/paper/2212.09248/) (2022)
- [LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completion](https://mlsanity.com/paper/2406.09834/) (2024)
- [Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks](https://mlsanity.com/paper/2102.12592/) (2021)
- [Bridging the Prototype-Production Gap: A Multi-Agent System for Notebooks Transformation](https://mlsanity.com/paper/2511.07257/) (2025)
- [Why do Machine Learning Notebooks Crash? An Empirical Study on Public Python Jupyter Notebooks](https://mlsanity.com/paper/2411.16795/) (2024)
- [Automated Assertion Generation and Regression Testing for Machine Learning Notebooks](https://mlsanity.com/paper/2509.13656/) (2025)

## Built on by (most cited first, 1 of 1)

- [Assembling Insights for Agentic Machine Learning Engineering Systems](https://mlsanity.com/paper/2610.04927/) (2026)

## Abstract (arXiv)

Interactive computational notebooks (e.g., Jupyter notebooks) are widely used in machine learning engineering (MLE) to program and share end-to-end pipelines, from data preparation to model training and evaluation. However, environmental erosion-the rapid evolution of hardware and software ecosystems for machine learning-has rendered many published MLE notebooks non-reproducible in contemporary environments, hindering code reuse and scientific progress. To quantify this gap, we study 12,106 notebooks selected from 75 popular Kaggle competitions: only 26% remain reproducible today. Crucially, we find that environment backporting, i.e., downgrading dependencies to match the submission time, does not improve reproducibility (decreased to 12%) but rather introduces additional failure modes. To address environmental erosion, we design and implement MLEModernizer, an LLM-driven agentic framework that treats the contemporary environment as a fixed constraint and modernizes notebook code to restore reproducibility. MLEModernizer iteratively executes notebooks, collects execution feedback, and applies three types of targeted fixes: error-repair, runtime-reduction, and score-calibration. Evaluated on 8,210 notebooks that are non-reproducible under the baseline environment, MLEModernizer makes 3,292 (40.1%, GPT-5.2) and 3,683 (44.9%, GPT-OSS-120b) notebooks reproducible. MLEModernizer presents a best-effort automated recovery and modernization technique that can improve reproducibility for a subset of notebooks. Practitioners can leverage MLEModernizer to validate, reuse, and maintain MLE artifacts as the hardware and software ecosystems continue to evolve.