# Reproducibility in machine learning for medical imaging

arXiv: https://arxiv.org/abs/2209.05097 · HTML: https://mlsanity.com/paper/2209.05097/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 82 references that are in the map.
- First posted on arXiv 12 Sep 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation](https://mlsanity.com/paper/1904.07773/) (2019)
- [Cross-validation failure: small sample sizes lead to large error bars](https://mlsanity.com/paper/1706.07581/) (2017)
- [Reproducibility in Machine Learning for Health](https://mlsanity.com/paper/1907.01463/) (2019)
- [pymia: A Python package for data handling and evaluation in deep learning-based medical image analysis](https://mlsanity.com/paper/2010.03639/) (2020)
- [Reproducible evaluation of classification methods in Alzheimer's disease: framework and application to MRI and PET data](https://mlsanity.com/paper/1808.06452/) (2018)
- [Automatic quality control of brain T1-weighted magnetic resonance images for a clinical data warehouse](https://mlsanity.com/paper/2104.08131/) (2021)
- [The MICCAI Hackathon on reproducibility, diversity, and selection of papers at the MICCAI conference](https://mlsanity.com/paper/2103.05437/) (2021)
- [Medical imaging data structure extended to multiple modalities and anatomical regions](https://mlsanity.com/paper/2010.00434/) (2020)

## Built on by (most cited first, 5 of 5)

- [Reproducibility in Machine Learning-Driven Research](https://mlsanity.com/paper/2307.10320/) (2023)
- [Comparative Analysis of ImageNet Pre-Trained Deep Learning Models and DINOv2 in Medical Imaging Classification](https://mlsanity.com/paper/2402.07595/) (2024)
- [Confidence Intervals for Performance Estimates in Brain MRI Segmentation](https://mlsanity.com/paper/2307.10926/) (2023)
- [Reliable Benchmarking of Artifact Detection in Computational Pathology: A Reproducibility and Uncertainty Analysis](https://mlsanity.com/paper/2608.30835/) (2026)
- [Radiuma: A Unified Zero-Code Executable Graphical Workflow Generator for Reproducible and Shareable Medical Image Analysis and Machine Learning](https://mlsanity.com/paper/2605.24201/) (2026)

## Abstract (arXiv)

Reproducibility is a cornerstone of science, as the replication of findings is the process through which they become knowledge. It is widely considered that many fields of science are undergoing a reproducibility crisis. This has led to the publications of various guidelines in order to improve research reproducibility. This didactic chapter intends at being an introduction to reproducibility for researchers in the field of machine learning for medical imaging. We first distinguish between different types of reproducibility. For each of them, we aim at defining it, at describing the requirements to achieve it and at discussing its utility. The chapter ends with a discussion on the benefits of reproducibility and with a plea for a non-dogmatic approach to this concept and its implementation in research practice.