mlsanityData to 7 Oct 2026

Automated Modernization of Machine Learning Engineering Notebooks for Reproducibility

arXiv 2602.07195 ↗First posted 6 Feb 2026cs.SE cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Interactive computational notebooks (e.g., Jupyter notebooks) are widely used in machine learning engineering (MLE) to program and share end-to-end pipelines, from data preparation to model training and evaluation. However, environmental erosion-the rapid evolution of hardware and software ecosystems for machine learning-has rendered many published MLE notebooks non-reproducible in contemporary environments, hindering code reuse and scientific progress. To quantify this gap, we study 12,106 notebooks selected from 75 popular Kaggle competitions: only 26% remain reproducible today. Crucially, we find that environment backporting, i.e., downgrading dependencies to match the submission time, does not improve reproducibility (decreased to 12%) but rather introduces additional failure modes. To address environmental erosion, we design and implement MLEModernizer, an LLM-driven agentic framework that treats the contemporary environment as a fixed constraint and modernizes notebook code to restore reproducibility. MLEModernizer iteratively executes notebooks, collects execution feedback, and applies three types of targeted fixes: error-repair, runtime-reduction, and score-calibration. Evaluated on 8,210 notebooks that are non-reproducible under the baseline environment, MLEModernizer makes 3,292 (40.1%, GPT-5.2) and 3,683 (44.9%, GPT-OSS-120b) notebooks reproducible. MLEModernizer presents a best-effort automated recovery and modernization technique that can improve reproducibility for a subset of notebooks. Practitioners can leverage MLEModernizer to validate, reuse, and maintain MLE artifacts as the hardware and software ecosystems continue to evolve.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 30 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

10 of 30 references.

References · 10

  1. 2017Deep Reinforcement Learning that Matterscited by 727
  2. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  3. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  4. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repaircited by 225
  5. 2022Natural Language to Code Generation in Interactive Data Science Notebookscited by 86
  6. 2024LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-based Code Completioncited by 29
  7. 2021Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebookscited by 17
  8. 2025Bridging the Prototype-Production Gap: A Multi-Agent System for Notebooks Transformationcited by 6
  9. 2024Why do Machine Learning Notebooks Crash? An Empirical Study on Public Python Jupyter Notebookscited by 4
  10. 2025Automated Assertion Generation and Regression Testing for Machine Learning Notebookscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues
  2. 2026DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories
  3. 2026Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool
  4. 2025On The Reproducibility Limitations of RAG Systems
  5. 2026RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?
  6. 2024Untangling Knots: Leveraging LLM for Error Resolution in Computational Notebooks
  7. 2025MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering
  8. 2025Reinforcement Learning for Machine Learning Engineering Agents