# On the Variance of Neural Network Training with respect to Test Sets and Distributions

arXiv: https://arxiv.org/abs/2304.01910 · HTML: https://mlsanity.com/paper/2304.01910/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 48 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Learning Robust Global Representations by Penalizing Local Predictive Power](https://mlsanity.com/paper/1905.13549/) (2019)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [Linear Mode Connectivity and the Lottery Ticket Hypothesis](https://mlsanity.com/paper/1912.05671/) (2019)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [ResNet strikes back: An improved training procedure in timm](https://mlsanity.com/paper/2110.00476/) (2021)
- [Essentially No Barriers in Neural Network Energy Landscape](https://mlsanity.com/paper/1803.00885/) (2018)
- [The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks](https://mlsanity.com/paper/2110.06296/) (2021)
- [The Loss Surfaces of Multilayer Networks](https://mlsanity.com/paper/1412.0233/) (2014)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [Convergent Learning: Do different neural networks learn the same representations?](https://mlsanity.com/paper/1511.07543/) (2015)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [Deep Deterministic Uncertainty: A Simple Baseline](https://mlsanity.com/paper/2102.11582/) (2021)
- [Datamodels: Predicting Predictions from Training Data](https://mlsanity.com/paper/2202.00622/) (2022)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [Deep Learning Through the Lens of Example Difficulty](https://mlsanity.com/paper/2106.09647/) (2021)
- [Assessing Generalization of SGD via Disagreement](https://mlsanity.com/paper/2106.13799/) (2021)
- [Optimizing Mode Connectivity via Neuron Alignment](https://mlsanity.com/paper/2009.02439/) (2020)
- [Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision](https://mlsanity.com/paper/2109.08203/) (2021)
- [Randomness In Neural Network Training: Characterizing The Impact of Tooling](https://mlsanity.com/paper/2106.11872/) (2021)

## Built on by (most cited first, 22 of 22)

- [Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs](https://mlsanity.com/paper/2309.07311/) (2023)
- [Pre-training under infinite compute](https://mlsanity.com/paper/2509.14786/) (2025)
- [MAGIC: Near-Optimal Data Attribution for Deep Learning](https://mlsanity.com/paper/2504.16430/) (2025)
- [94% on CIFAR-10 in 3.29 Seconds on a Single GPU](https://mlsanity.com/paper/2404.00498/) (2024)
- [Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitions](https://mlsanity.com/paper/2401.04305/) (2024)
- [Random Scaling of Emergent Capabilities](https://mlsanity.com/paper/2502.17356/) (2025)
- [Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis](https://mlsanity.com/paper/2308.00404/) (2023)
- [Transferring Learning Trajectories of Neural Networks](https://mlsanity.com/paper/2305.14122/) (2023)
- [Simple Projection Variants Improve ColBERT Performance](https://mlsanity.com/paper/2510.12327/) (2025)
- [The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation](https://mlsanity.com/paper/2606.20536/) (2026)
- [Robust Nonparametric Hypothesis Testing to Understand Variability in Training Neural Networks](https://mlsanity.com/paper/2310.00541/) (2023)
- [On Arbitrary Predictions from Equally Valid Models](https://mlsanity.com/paper/2507.19408/) (2025)
- [VICTOR: Dataset Copyright Auditing in Video Recognition Systems](https://mlsanity.com/paper/2512.14439/) (2025)
- [Model Agreement via Anchoring](https://mlsanity.com/paper/2602.23360/) (2026)
- [Intrinsic Barriers to Explaining Deep Foundation Models](https://mlsanity.com/paper/2504.16948/) (2025)
- [Beginning with You: Perceptual-Initialization Improves Vision-Language Representation and Alignment](https://mlsanity.com/paper/2505.14204/) (2025)
- [f-INE: A Hypothesis Testing Framework for Estimating Influence under Training Randomness](https://mlsanity.com/paper/2510.10510/) (2025)
- [Sequence of Expert: Boosting Imitation Planners for Autonomous Driving through Temporal Alternation](https://mlsanity.com/paper/2512.13094/) (2025)
- [Adversarial Samples Are Not Created Equal](https://mlsanity.com/paper/2601.00577/) (2026)
- [On the Extreme Variance of Certified Local Robustness Across Model Seeds](https://mlsanity.com/paper/2601.13303/) (2026)
- [Bit-Identical Medical Deep Learning via Structured Orthogonal Initialization](https://mlsanity.com/paper/2603.28040/) (2026)
- [Measuring consistency via ensemble margin and local prediction variability: Auditing decision systems in the presence of predictive multiplicity](https://mlsanity.com/paper/2609.01397/) (2026)

## Abstract (arXiv)

Typical neural network trainings have substantial variance in test-set performance between repeated runs, impeding hyperparameter comparison and training reproducibility. In this work we present the following results towards understanding this variation. (1) Despite having significant variance on their test-sets, we demonstrate that standard CIFAR-10 and ImageNet trainings have little variance in performance on the underlying test-distributions from which their test-sets are sampled. (2) We show that these trainings make approximately independent errors on their test-sets. That is, the event that a trained network makes an error on one particular example does not affect its chances of making errors on other examples, relative to their average rates over repeated runs of training with the same hyperparameters. (3) We prove that the variance of neural network trainings on their test-sets is a downstream consequence of the class-calibration property discovered by Jiang et al. (2021). Our analysis yields a simple formula which accurately predicts variance for the binary classification case. (4) We conduct preliminary studies of data augmentation, learning rate, finetuning instability and distribution-shift through the lens of variance between runs.