# Quantifying Inherent Randomness in Machine Learning Algorithms

arXiv: https://arxiv.org/abs/2206.12353 · HTML: https://mlsanity.com/paper/2206.12353/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 12 references that are in the map.
- First posted on arXiv 24 Jun 2022.

## Builds on (most cited first, 7 of 7)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice](https://mlsanity.com/paper/2007.15745/) (2020)
- [On Model Stability as a Function of Random Seed](https://mlsanity.com/paper/1909.10447/) (2019)
- [On the Reproducibility of Neural Network Predictions](https://mlsanity.com/paper/2102.03349/) (2021)
- [Supervised Machine Learning Techniques: An Overview with Applications to Banking](https://mlsanity.com/paper/2008.04059/) (2020)
- [Performance and Interpretability Comparisons of Supervised Machine Learning Algorithms: An Empirical Study](https://mlsanity.com/paper/2204.12868/) (2022)

## Built on by (most cited first, 6 of 6)

- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers](https://mlsanity.com/paper/2406.14325/) (2024)
- [Transfer Learning in ECG Diagnosis: Is It Effective?](https://mlsanity.com/paper/2402.02021/) (2024)
- [Investigating the Impact of Randomness on Reproducibility in Computer Vision: A Study on Applications in Civil Engineering and Medicine](https://mlsanity.com/paper/2410.02806/) (2024)
- [Measuring Neural Network Complexity via Effective Degrees of Freedom](https://mlsanity.com/paper/2602.13442/) (2026)
- [A Novel Method to Determine Total Oxidant Concentration Produced by Non-Thermal Plasma Based on Image Processing and Machine Learning](https://mlsanity.com/paper/2509.00479/) (2025)

## Abstract (arXiv)

Most machine learning (ML) algorithms have several stochastic elements, and their performances are affected by these sources of randomness. This paper uses an empirical study to systematically examine the effects of two sources: randomness in model training and randomness in the partitioning of a dataset into training and test subsets. We quantify and compare the magnitude of the variation in predictive performance for the following ML algorithms: Random Forests (RFs), Gradient Boosting Machines (GBMs), and Feedforward Neural Networks (FFNNs). Among the different algorithms, randomness in model training causes larger variation for FFNNs compared to tree-based methods. This is to be expected as FFNNs have more stochastic elements that are part of their model initialization and training. We also found that random splitting of datasets leads to higher variation compared to the inherent randomness from model training. The variation from data splitting can be a major issue if the original dataset has considerable heterogeneity. Keywords: Model Training, Reproducibility, Variation