# We need to talk about random seeds

arXiv: https://arxiv.org/abs/2210.13393 · HTML: https://mlsanity.com/paper/2210.13393/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 26 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [Show Your Work: Improved Reporting of Experimental Results](https://mlsanity.com/paper/1909.03004/) (2019)
- [Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging](https://mlsanity.com/paper/1707.09861/) (2017)
- [Noise Stability Regularization for Improving BERT Fine-tuning](https://mlsanity.com/paper/2107.04835/) (2021)
- [Syntactic Structure Distillation Pretraining For Bidirectional Encoders](https://mlsanity.com/paper/2005.13482/) (2020)

## Built on by (most cited first, 13 of 13)

- [On The Impact of Machine Learning Randomness on Group Fairness](https://mlsanity.com/paper/2307.04138/) (2023)
- [Assessing the Macro and Micro Effects of Random Seeds on Fine-Tuning Large Language Models](https://mlsanity.com/paper/2503.07329/) (2025)
- [Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility](https://mlsanity.com/paper/2608.04001/) (2026)
- [Better than Random: Reliable NLG Human Evaluation with Constrained Active Sampling](https://mlsanity.com/paper/2406.07967/) (2024)
- [Explanation sensitivity to the randomness of large language models: the case of journalistic text classification](https://mlsanity.com/paper/2410.05085/) (2024)
- [Simple Projection Variants Improve ColBERT Performance](https://mlsanity.com/paper/2510.12327/) (2025)
- [Beyond statistical significance: Quantifying uncertainty and statistical variability in multilingual and multitask NLP evaluation](https://mlsanity.com/paper/2509.22612/) (2025)
- [Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence](https://mlsanity.com/paper/2608.12290/) (2026)
- [A Cascaded Neural Network System For Rating Student Performance In Surgical Knot Tying Simulation](https://mlsanity.com/paper/2312.14952/) (2023)
- [The Costs of Reproducibility in Music Separation Research: a Replication of Band-Split RNN](https://mlsanity.com/paper/2603.09187/) (2026)
- [Dynamic Scaled Gradient Descent for Stable Fine-Tuning for Classifications](https://mlsanity.com/paper/2604.27987/) (2026)
- [Gradient-Descent Steps to Success over Mean Accuracy: A Paradigm Shift for ML](https://mlsanity.com/paper/2606.22053/) (2026)
- [GRAIN: Group Aggregation via Min-Norm Objective](https://mlsanity.com/paper/2606.22917/) (2026)

## Abstract (arXiv)

Modern neural network libraries all take as a hyperparameter a random seed, typically used to determine the initial state of the model parameters. This opinion piece argues that there are some safe uses for random seeds: as part of the hyperparameter search to select a good model, creating an ensemble of several models, or measuring the sensitivity of the training algorithm to the random seed hyperparameter. It argues that some uses for random seeds are risky: using a fixed random seed for "replicability" and varying only the random seed to create score distributions for performance comparison. An analysis of 85 recent publications from the ACL Anthology finds that more than 50% contain risky uses of random seeds.