# Sources of Irreproducibility in Machine Learning: A Review

arXiv: https://arxiv.org/abs/2204.07610 · HTML: https://mlsanity.com/paper/2204.07610/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 103 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 25 of 25)

- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Are GANs Created Equal? A Large-Scale Study](https://mlsanity.com/paper/1711.10337/) (2017)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Show Your Work: Improved Reporting of Experimental Results](https://mlsanity.com/paper/1909.03004/) (2019)
- [Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans](https://mlsanity.com/paper/2008.06388/) (2020)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [Troubling Trends in Machine Learning Scholarship](https://mlsanity.com/paper/1807.03341/) (2018)
- [On the State of the Art of Evaluation in Neural Language Models](https://mlsanity.com/paper/1707.05589/) (2017)
- [A Step Toward Quantifying Independently Reproducible Machine Learning Research](https://mlsanity.com/paper/1909.06674/) (2019)
- [Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging](https://mlsanity.com/paper/1707.09861/) (2017)
- [A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research](https://mlsanity.com/paper/1911.07698/) (2019)
- [Randomness In Neural Network Training: Characterizing The Impact of Tooling](https://mlsanity.com/paper/2106.11872/) (2021)
- [A Systematic Review of Reproducibility Research in Natural Language Processing](https://mlsanity.com/paper/2103.07929/) (2021)
- [A Large-Scale Study on Regularization and Normalization in GANs](https://mlsanity.com/paper/1807.04720/) (2018)
- [Hyperparameter Optimization Is Deceiving Us, and How to Stop It](https://mlsanity.com/paper/2102.03034/) (2021)
- [The Fundamental Principles of Reproducibility](https://mlsanity.com/paper/2011.10098/) (2020)
- [The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning](https://mlsanity.com/paper/2203.06498/) (2022)
- [Research Reproducibility as a Survival Analysis](https://mlsanity.com/paper/2012.09932/) (2020)
- [Session-aware Recommendation: A Surprising Quest for the State-of-the-art](https://mlsanity.com/paper/2011.03424/) (2020)
- [Critical analysis on the reproducibility of visual quality assessment using deep features](https://mlsanity.com/paper/2009.05369/) (2020)
- [On the reproducibility of fully convolutional neural networks for modeling time-space evolving physical systems](https://mlsanity.com/paper/2105.05482/) (2021)

## Built on by (most cited first, 30 of 32)

- [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024)
- [Efficient Methods for Natural Language Processing: A Survey](https://mlsanity.com/paper/2209.00099/) (2022)
- [Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers](https://mlsanity.com/paper/2406.14325/) (2024)
- [Benchmark Data Repositories for Better Benchmarking](https://mlsanity.com/paper/2410.24100/) (2024)
- [EBES: Easy Benchmarking for Event Sequences](https://mlsanity.com/paper/2410.03399/) (2024)
- [A Two-Sided Discussion of Preregistration of NLP Research](https://mlsanity.com/paper/2302.10086/) (2023)
- [Towards Inferential Reproducibility of Machine Learning Research](https://mlsanity.com/paper/2302.04054/) (2023)
- [The Impact of Feature Scaling In Machine Learning: Effects on Regression and Classification Tasks](https://mlsanity.com/paper/2506.08274/) (2025)
- [Examining the Effect of Implementation Factors on Deep Learning Reproducibility](https://mlsanity.com/paper/2312.06633/) (2023)
- [Towards a Categorical Foundation of Deep Learning: A Survey](https://mlsanity.com/paper/2410.05353/) (2024)
- [AI Copilots for Reproducibility in Science: A Case Study](https://mlsanity.com/paper/2506.20130/) (2025)
- [How Open Must Language Models be to Enable Reliable Scientific Inference?](https://mlsanity.com/paper/2603.26539/) (2026)
- [Online learning techniques for prediction of temporal tabular datasets with regime changes](https://mlsanity.com/paper/2301.00790/) (2022)
- [A Rigorous Uncertainty-Aware Quantification Framework Is Essential for Reproducible and Replicable Machine Learning Workflows](https://mlsanity.com/paper/2301.05763/) (2023)
- [Deep incremental learning models for financial temporal tabular datasets with distribution shifts](https://mlsanity.com/paper/2303.07925/) (2023)
- [Robust Nonparametric Hypothesis Testing to Understand Variability in Training Neural Networks](https://mlsanity.com/paper/2310.00541/) (2023)
- [Measuring training variability from stochastic optimization using robust nonparametric testing](https://mlsanity.com/paper/2406.08307/) (2024)
- [The Unreasonable Effectiveness of Open Science in AI: A Replication Study](https://mlsanity.com/paper/2412.17859/) (2024)
- [Statistical Quality and Reproducibility of Pseudorandom Number Generators in Machine Learning technologies](https://mlsanity.com/paper/2507.03007/) (2025)
- [The Shift Toward Open and Reproducible AI Research](https://mlsanity.com/paper/2606.16974/) (2026)
- [An End-to-End System for Reproducibility Assessment of Source Code Repositories via Their Readmes](https://mlsanity.com/paper/2310.09634/) (2023)
- [A Second Look on BASS -- Boosting Abstractive Summarization with Unified Semantic Graphs -- A Replication Study](https://mlsanity.com/paper/2403.02930/) (2024)
- [Reproducibility and Geometric Intrinsic Dimensionality: An Investigation on Graph Neural Network Research](https://mlsanity.com/paper/2403.08438/) (2024)
- [Generalizability of experimental studies](https://mlsanity.com/paper/2406.17374/) (2024)
- [Aerial Image Classification in Scarce and Unconstrained Environments via Conformal Prediction](https://mlsanity.com/paper/2504.17655/) (2025)
- [Reducing Variability of Multiple Instance Learning Methods for Digital Pathology](https://mlsanity.com/paper/2507.00292/) (2025)
- [Fractal basin geometry and the limits to predictability and reproducibility in deep learning](https://mlsanity.com/paper/2510.05606/) (2025)
- [Position: Ideas Should be the Center of Machine Learning Research](https://mlsanity.com/paper/2605.15253/) (2026)
- [I-CARE: Analysis of interference-related phenomena in a controllable, diverse and representative unlearning setting for text-to-image models](https://mlsanity.com/paper/2609.00003/) (2026)
- [Measuring consistency via ensemble margin and local prediction variability: Auditing decision systems in the presence of predictive multiplicity](https://mlsanity.com/paper/2609.01397/) (2026)

## Abstract (arXiv)

Background: Many published machine learning studies are irreproducible. Issues with methodology and not properly accounting for variation introduced by the algorithm themselves or their implementations are attributed as the main contributors to the irreproducibility.Problem: There exist no theoretical framework that relates experiment design choices to potential effects on the conclusions. Without such a framework, it is much harder for practitioners and researchers to evaluate experiment results and describe the limitations of experiments. The lack of such a framework also makes it harder for independent researchers to systematically attribute the causes of failed reproducibility experiments. Objective: The objective of this paper is to develop a framework that enable applied data science practitioners and researchers to understand which experiment design choices can lead to false findings and how and by this help in analyzing the conclusions of reproducibility experiments. Method: We have compiled an extensive list of factors reported in the literature that can lead to machine learning studies being irreproducible. These factors are organized and categorized in a reproducibility framework motivated by the stages of the scientific method. The factors are analyzed for how they can affect the conclusions drawn from experiments. A model comparison study is used as an example. Conclusion: We provide a framework that describes machine learning methodology from experimental design decisions to the conclusions inferred from them.