# On the Reproducibility of Neural Network Predictions

arXiv: https://arxiv.org/abs/2102.03349 · HTML: https://mlsanity.com/paper/2102.03349/

## Key facts

- Cited by 28 papers in the map (data through 7 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 5 Feb 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning](https://mlsanity.com/paper/1412.6614/) (2014)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Knowledge Distillation by On-the-Fly Native Ensemble](https://mlsanity.com/paper/1806.04606/) (2018)
- [Large scale distributed neural network training through online distillation](https://mlsanity.com/paper/1804.03235/) (2018)
- [Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals](https://mlsanity.com/paper/1809.04198/) (2018)
- [Collaborative Learning for Deep Neural Networks](https://mlsanity.com/paper/1805.11761/) (2018)
- [Satisfying Real-world Goals with Dataset Constraints](https://mlsanity.com/paper/1606.07558/) (2016)
- [On Model Stability as a Function of Random Seed](https://mlsanity.com/paper/1909.10447/) (2019)
- [Non-Determinism in TensorFlow ResNets](https://mlsanity.com/paper/2001.11396/) (2020)
- [Ten Simple Rules for Reproducible Research in Jupyter Notebooks](https://mlsanity.com/paper/1810.08055/) (2018)
- [Open science in machine learning](https://mlsanity.com/paper/1402.6013/) (2014)

## Built on by (most cited first, 28 of 28)

- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [Assessing Generalization of SGD via Disagreement](https://mlsanity.com/paper/2106.13799/) (2021)
- [Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits](https://mlsanity.com/paper/2411.18704/) (2024)
- [On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models](https://mlsanity.com/paper/2209.05310/) (2022)
- [Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions](https://mlsanity.com/paper/2202.01331/) (2022)
- [Reproducibility in Optimization: Theoretical Framework and Limits](https://mlsanity.com/paper/2202.04598/) (2022)
- [On the Variance of Neural Network Training with respect to Test Sets and Distributions](https://mlsanity.com/paper/2304.01910/) (2023)
- [Churn Reduction via Distillation](https://mlsanity.com/paper/2106.02654/) (2021)
- [Neural Network Weights Do Not Converge to Stationary Points: An Invariant Measure Perspective](https://mlsanity.com/paper/2110.06256/) (2021)
- [Synthesizing Irreproducibility in Deep Networks](https://mlsanity.com/paper/2102.10696/) (2021)
- [Measuring the Instability of Fine-Tuning](https://mlsanity.com/paper/2302.07778/) (2023)
- [Predicting on the Edge: Identifying Where a Larger Model Does Better](https://mlsanity.com/paper/2202.07652/) (2022)
- [On the Prediction Instability of Graph Neural Networks](https://mlsanity.com/paper/2205.10070/) (2022)
- [Quantifying Inherent Randomness in Machine Learning Algorithms](https://mlsanity.com/paper/2206.12353/) (2022)
- [Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations](https://mlsanity.com/paper/2202.06499/) (2022)
- [Transferring Learning Trajectories of Neural Networks](https://mlsanity.com/paper/2305.14122/) (2023)
- [Maintaining Stability and Plasticity for Predictive Churn Reduction](https://mlsanity.com/paper/2305.04135/) (2023)
- [Measuring and Mitigating Local Instability in Deep Neural Networks](https://mlsanity.com/paper/2305.10625/) (2023)
- [PIDSMaker: Building and Evaluating Provenance-based Intrusion Detection Systems](https://mlsanity.com/paper/2601.22983/) (2026)
- [Model Agreement via Anchoring](https://mlsanity.com/paper/2602.23360/) (2026)
- [Reducing cross-sample prediction churn in scientific machine learning](https://mlsanity.com/paper/2605.13826/) (2026)
- [Convex Dual Theory Analysis of Two-Layer Convolutional Neural Networks with Soft-Thresholding](https://mlsanity.com/paper/2304.06959/) (2023)
- [Distilling Influences to Mitigate Prediction Churn in Graph Neural Networks](https://mlsanity.com/paper/2310.00946/) (2023)
- [Fractal basin geometry and the limits to predictability and reproducibility in deep learning](https://mlsanity.com/paper/2510.05606/) (2025)
- [Trustworthy AI: Ensuring Reliability and Accountability from Models to Agents](https://mlsanity.com/paper/2605.08964/) (2026)
- [From Accuracy to Auditability: A Survey of Determinism in Financial AI Systems](https://mlsanity.com/paper/2605.23955/) (2026)
- [Omega-S: A Functional Resilience Index for LLM Fine-Tuning](https://mlsanity.com/paper/2608.03887/) (2026)
- [How Divergence Becomes Decision Flips in Compressed Language Models](https://mlsanity.com/paper/2610.00694/) (2026)

## Abstract (arXiv)

Standard training techniques for neural networks involve multiple sources of randomness, e.g., initialization, mini-batch ordering and in some cases data augmentation. Given that neural networks are heavily over-parameterized in practice, such randomness can cause em churn -- for the same input, disagreements between predictions of the two models independently trained by the same algorithm, contributing to the `reproducibility challenges' in modern machine learning. In this paper, we study this problem of churn, identify factors that cause it, and propose two simple means of mitigating it. We first demonstrate that churn is indeed an issue, even for standard image classification tasks (CIFAR and ImageNet), and study the role of the different sources of training randomness that cause churn. By analyzing the relationship between churn and prediction confidences, we pursue an approach with two components for churn reduction. First, we propose using minimum entropy regularizers to increase prediction confidences. Second, changeswe present a novel variant of co-distillation approach~citepanil2018large to increase model agreement and reduce churn. We present empirical results showing the effectiveness of both techniques in reducing churn while improving the accuracy of the underlying model.