# Reproducibility, Replicability, and Repeatability: A survey of reproducible research with a focus on high performance computing

arXiv: https://arxiv.org/abs/2402.07530 · HTML: https://mlsanity.com/paper/2402.07530/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2024.

## Built on by (most cited first, 8 of 8)

- [Can Coding Agents Reproduce Findings in Computational Materials Science?](https://mlsanity.com/paper/2605.00803/) (2026)
- [LLM Output Drift: Cross-Provider Validation & Mitigation for Financial Workflows](https://mlsanity.com/paper/2511.07585/) (2025)
- [Statistical Quality and Reproducibility of Pseudorandom Number Generators in Machine Learning technologies](https://mlsanity.com/paper/2507.03007/) (2025)
- [CC30k: A Citation Contexts Dataset for Reproducibility-Oriented Sentiment Analysis](https://mlsanity.com/paper/2511.07790/) (2025)
- [A K-Means, Ward and DBSCAN repeatability study](https://mlsanity.com/paper/2512.19772/) (2025)
- [Reproducible AI Requires Reproducible Randomness](https://mlsanity.com/paper/2609.26461/) (2026)
- [Position: Let's Strengthen Verifiability If We Can't Enforce Reproducibility](https://mlsanity.com/paper/2609.35854/) (2026)
- [The Other Half of Workflow Portability: Evidence-Backed HPC Site Profiles with Agentic Discovery](https://mlsanity.com/paper/2610.00971/) (2026)

## Abstract (arXiv)

Reproducibility is widely acknowledged as a fundamental principle in scientific research. Currently, the scientific community grapples with numerous challenges associated with reproducibility, often referred to as the ''reproducibility crisis.'' This crisis permeated numerous scientific disciplines. In this study, we examined the factors in scientific practices that might contribute to this lack of reproducibility. Significant focus is placed on the prevalent integration of computation in research, which can sometimes function as a black box in published papers. Our study primarily focuses on highperformance computing (HPC), which presents unique reproducibility challenges. This paper provides a comprehensive review of these concerns and potential solutions. Furthermore, we discuss the critical role of reproducible research in advancing science and identifying persisting issues within the field of HPC.