mlsanityData to 7 Oct 2026

Perspectives on Machine Learning from Psychology's Reproducibility Crisis

arXiv 2104.08878 ↗First posted 18 Apr 2021cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

In the early 2010s, a crisis of reproducibility rocked the field of psychology. Following a period of reflection, the field has responded with radical reform of its scientific practices. More recently, similar questions about the reproducibility of machine learning research have also come to the fore. In this short paper, we present select ideas from psychology's reformation, translating them into relevance for a machine learning audience.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 7 of its 38 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

7 of 38 references.

References · 7

  1. 2017Deep Reinforcement Learning that Matterscited by 727
  2. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  3. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 164
  4. 2018Troubling Trends in Machine Learning Scholarshipcited by 107
  5. 2017On the State of the Art of Evaluation in Neural Language Modelscited by 103
  6. 2017Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Taggingcited by 71
  7. 2019The Scientific Method in the Science of Machine Learningcited by 15

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 4; 2023: 2; 2024: 1; 2025: 1.

9 papers.

  1. 2022Leakage and the Reproducibility Crisis in ML-based Sciencecited by 63
  2. 2023Sensitivity-Aware Amortized Bayesian Inferencecited by 20
  3. 2022The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learningcited by 17
  4. 2023REFORMS: Reporting Standards for Machine Learning Based Sciencecited by 13
  5. 2022Modeling the Machine Learning Multiversecited by 11
  6. 2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asiacited by 8
  7. 2022Prescriptive and Descriptive Approaches to Machine-Learning Transparencycited by 7
  8. 2021Behavioral Experiments for Understanding Catastrophic Forgettingcited by 2
  9. 2024On the Role of Speech Data in Reducing Toxicity Detection Biascited by 2