mlsanityData to 7 Oct 2026

The Fundamental Principles of Reproducibility

arXiv 2011.10098 ↗First posted 19 Nov 2020cs.LG cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

Reproducibility is a confused terminology. In this paper, I take a fundamental view on reproducibility rooted in the scientific method. The scientific method is analysed and characterised in order to develop the terminology required to define reproducibility. Further, the literature on reproducibility and replication is surveyed, and experiments are modeled as tasks and problem solving methods. Machine learning is used to exemplify the described approach. Based on the analysis, reproducibility is defined and three different types of reproducibility as well as four degrees of reproducibility are specified.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 7; 2025: 5; 2026: 1.

17 papers.

Sort:

By area:

  1. 2024Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equationscited by 87
  2. 2022Leakage and the Reproducibility Crisis in ML-based Sciencecited by 63
  3. 2024Reproducibility in Machine Learning-based Research: Overview, Barriers and Driverscited by 34
  4. 2022Sources of Irreproducibility in Machine Learning: A Reviewcited by 28
  5. 2024Position: Why We Must Rethink Empirical Research in Machine Learningcited by 19
  6. 2025Reflections on the Reproducibility of Commercial LLM Performance in Empirical Software Engineering Studiescited by 12
  7. 2024What is Reproducibility in Artificial Intelligence and Machine Learning Research?cited by 11
  8. 2024Benchmark Data Repositories for Better Benchmarkingcited by 10
  9. 2023Reproducibility of Machine Learning: Terminology, Recommendations and Open Issuescited by 8
  10. 2025GENIUS: An Agentic AI Framework for Autonomous Design and Execution of Simulation Protocolscited by 4
  11. 2023Examining the Effect of Implementation Factors on Deep Learning Reproducibilitycited by 3
  12. 2025Modeling the Economic Impacts of AI Openness Regulationcited by 3
5 more
  1. 2024From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gapcited by 1
  2. 2024Reproducibility and Geometric Intrinsic Dimensionality: An Investigation on Graph Neural Network Researchcited by 0
  3. 2025From Data to Decision: Data-Centric Infrastructure for Reproducible ML in Collaborative eSciencecited by 0
  4. 2025A K-Means, Ward and DBSCAN repeatability studycited by 0
  5. 2026RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?cited by 0