# Perspectives on Machine Learning from Psychology's Reproducibility Crisis

arXiv: https://arxiv.org/abs/2104.08878 · HTML: https://mlsanity.com/paper/2104.08878/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 38 references that are in the map.
- First posted on arXiv 18 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Show Your Work: Improved Reporting of Experimental Results](https://mlsanity.com/paper/1909.03004/) (2019)
- [Troubling Trends in Machine Learning Scholarship](https://mlsanity.com/paper/1807.03341/) (2018)
- [On the State of the Art of Evaluation in Neural Language Models](https://mlsanity.com/paper/1707.05589/) (2017)
- [Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging](https://mlsanity.com/paper/1707.09861/) (2017)
- [The Scientific Method in the Science of Machine Learning](https://mlsanity.com/paper/1904.10922/) (2019)

## Built on by (most cited first, 9 of 9)

- [Leakage and the Reproducibility Crisis in ML-based Science](https://mlsanity.com/paper/2207.07048/) (2022)
- [Sensitivity-Aware Amortized Bayesian Inference](https://mlsanity.com/paper/2310.11122/) (2023)
- [The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning](https://mlsanity.com/paper/2203.06498/) (2022)
- [REFORMS: Reporting Standards for Machine Learning Based Science](https://mlsanity.com/paper/2308.07832/) (2023)
- [Modeling the Machine Learning Multiverse](https://mlsanity.com/paper/2206.05985/) (2022)
- [Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asia](https://mlsanity.com/paper/2503.07920/) (2025)
- [Prescriptive and Descriptive Approaches to Machine-Learning Transparency](https://mlsanity.com/paper/2204.13582/) (2022)
- [Behavioral Experiments for Understanding Catastrophic Forgetting](https://mlsanity.com/paper/2110.10570/) (2021)
- [On the Role of Speech Data in Reducing Toxicity Detection Bias](https://mlsanity.com/paper/2411.08135/) (2024)

## Abstract (arXiv)

In the early 2010s, a crisis of reproducibility rocked the field of psychology. Following a period of reflection, the field has responded with radical reform of its scientific practices. More recently, similar questions about the reproducibility of machine learning research have also come to the fore. In this short paper, we present select ideas from psychology's reformation, translating them into relevance for a machine learning audience.