# Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision

arXiv: https://arxiv.org/abs/2109.08203 · HTML: https://mlsanity.com/paper/2109.08203/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 4 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 3 of 3)

- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Accounting for Variance in Machine Learning Benchmarks](https://mlsanity.com/paper/2103.03098/) (2021)
- [Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark](https://mlsanity.com/paper/1806.01427/) (2018)

## Built on by (most cited first, 30 of 81)

- [ResNet strikes back: An improved training procedure in timm](https://mlsanity.com/paper/2110.00476/) (2021)
- [DiffusionDet: Diffusion Model for Object Detection](https://mlsanity.com/paper/2211.09788/) (2022)
- [Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2405.14828/) (2024)
- [Building Production-Ready Probes For Gemini](https://mlsanity.com/paper/2601.11516/) (2026)
- [If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selection](https://mlsanity.com/paper/2305.13308/) (2023)
- [Augmenting Convolutional networks with attention-based aggregation](https://mlsanity.com/paper/2112.13692/) (2021)
- [Learned Queries for Efficient Local Attention](https://mlsanity.com/paper/2112.11435/) (2021)
- [On the Variance of Neural Network Training with respect to Test Sets and Distributions](https://mlsanity.com/paper/2304.01910/) (2023)
- [The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning](https://mlsanity.com/paper/2203.06498/) (2022)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [Instant Adversarial Purification with Adversarial Consistency Distillation](https://mlsanity.com/paper/2408.17064/) (2024)
- [Probing Deep into Temporal Profile Makes the Infrared Small Target Detector Much Better](https://mlsanity.com/paper/2506.12766/) (2025)
- [Artwork Protection Against Neural Style Transfer Using Locally Adaptive Adversarial Color Attack](https://mlsanity.com/paper/2401.09673/) (2024)
- [Assessing the Macro and Micro Effects of Random Seeds on Fine-Tuning Large Language Models](https://mlsanity.com/paper/2503.07329/) (2025)
- [SOS! Self-supervised Learning Over Sets Of Handled Objects In Egocentric Action Recognition](https://mlsanity.com/paper/2204.04796/) (2022)
- [Improving Deep Neural Network Random Initialization Through Neuronal Rewiring](https://mlsanity.com/paper/2207.08148/) (2022)
- [Multi-Loco: Unifying Multi-Embodiment Legged Locomotion via Reinforcement Learning Augmented Diffusion](https://mlsanity.com/paper/2506.11470/) (2025)
- [Confident AI](https://mlsanity.com/paper/2202.05957/) (2022)
- [CASS: Cross Architectural Self-Supervision for Medical Image Analysis](https://mlsanity.com/paper/2206.04170/) (2022)
- [A Data-Efficient Deep Learning Framework for Segmentation and Classification of Histopathology Images](https://mlsanity.com/paper/2207.06489/) (2022)
- [Position: Embracing Negative Results in Machine Learning](https://mlsanity.com/paper/2406.03980/) (2024)
- [Triamese-ViT: A 3D-Aware Method for Robust Brain Age Estimation from MRIs](https://mlsanity.com/paper/2401.09475/) (2024)
- [Positional encoding is not the same as context: A study on positional encoding for sequential recommendation](https://mlsanity.com/paper/2405.10436/) (2024)
- [Questionable practices in machine learning](https://mlsanity.com/paper/2407.12220/) (2024)
- [Revisiting Batch Norm Initialization](https://mlsanity.com/paper/2110.13989/) (2021)
- [Learning More May Not Be Better: Knowledge Transferability in Vision and Language Tasks](https://mlsanity.com/paper/2208.10758/) (2022)
- [Differentiable SVD based on Moore-Penrose Pseudoinverse for Inverse Imaging Problems](https://mlsanity.com/paper/2411.14141/) (2024)
- [Engineering the Neural Automatic Passenger Counter](https://mlsanity.com/paper/2203.01156/) (2022)
- [Merging of neural networks](https://mlsanity.com/paper/2204.09973/) (2022)
- [Efficient Representation Learning for Healthcare with Cross-Architectural Self-Supervision](https://mlsanity.com/paper/2308.10064/) (2023)

## Abstract (arXiv)

In this paper I investigate the effect of random seed selection on the accuracy when using popular deep learning architectures for computer vision. I scan a large amount of seeds (up to 104) on CIFAR 10 and I also scan fewer seeds on Imagenet using pre-trained models to investigate large scale datasets. The conclusions are that even if the variance is not very large, it is surprisingly easy to find an outlier that performs much better or much worse than the average.