mlsanityData to 7 Oct 2026

We need to talk about random seeds

arXiv 2210.13393 ↗First posted 24 Oct 2022cs.CL cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Modern neural network libraries all take as a hyperparameter a random seed, typically used to determine the initial state of the model parameters. This opinion piece argues that there are some safe uses for random seeds: as part of the hyperparameter search to select a good model, creating an ensemble of several models, or measuring the sensitivity of the training algorithm to the random seed hyperparameter. It argues that some uses for random seeds are risky: using a fixed random seed for "replicability" and varying only the random seed to create score distributions for performance comparison. An analysis of 85 recent publications from the ACL Anthology finds that more than 50% contain risky uses of random seeds.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 12 of its 26 references that are in the map. First posted on arXiv 24 Oct 2022.

Builds on

12 of 26 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2020Designing Network Design Spacescited by 954
  5. 2019Attention is not not Explanationcited by 629
  6. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  7. 2018How Does Batch Normalization Help Optimization?cited by 355
  8. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 254
  9. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 164
  10. 2017Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Taggingcited by 71
  11. 2021Noise Stability Regularization for Improving BERT Fine-tuningcited by 29
  12. 2020Syntactic Structure Distillation Pretraining For Bidirectional Encoderscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 3; 2026: 5.

12 papers.

  1. 2023On The Impact of Machine Learning Randomness on Group Fairnesscited by 26
  2. 2025Assessing the Macro and Micro Effects of Random Seeds on Fine-Tuning Large Language Modelscited by 10
  3. 2024Better than Random: Reliable NLG Human Evaluation with Constrained Active Samplingcited by 4
  4. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibilitycited by 4
  5. 2024Explanation sensitivity to the randomness of large language models: the case of journalistic text classificationcited by 3
  6. 2025Simple Projection Variants Improve ColBERT Performancecited by 2
  7. 2026Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherencecited by 1
  8. 2023A Cascaded Neural Network System For Rating Student Performance In Surgical Knot Tying Simulationcited by 0
  9. 2025Beyond statistical significance: Quantifying uncertainty and statistical variability in multilingual and multitask NLP evaluationcited by 0
  10. 2026The Costs of Reproducibility in Music Separation Research: a Replication of Band-Split RNNcited by 0
  11. 2026Dynamic Scaled Gradient Descent for Stable Fine-Tuning for Classificationscited by 0
  12. 2026GRAIN: Group Aggregation via Min-Norm Objectivecited by 0