# A Random CNN Sees Objects: One Inductive Bias of CNN and Its Applications

arXiv: https://arxiv.org/abs/2106.09259 · HTML: https://mlsanity.com/paper/2106.09259/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 42 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation](https://mlsanity.com/paper/2012.07177/) (2020)
- [Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection](https://mlsanity.com/paper/1708.01642/) (2017)
- [Proving the Lottery Ticket Hypothesis: Pruning is All You Need](https://mlsanity.com/paper/2002.00585/) (2020)
- [The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models](https://mlsanity.com/paper/2012.06908/) (2020)
- [Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2003.05438/) (2020)
- [Inductive Bias of Deep Convolutional Networks through Pooling Geometry](https://mlsanity.com/paper/1605.06743/) (2016)
- [Rethinking the Route Towards Weakly Supervised Object Localization](https://mlsanity.com/paper/2002.11359/) (2020)

## Built on by (most cited first, 15 of 15)

- [DisWOT: Student Architecture Search for Distillation WithOut Training](https://mlsanity.com/paper/2303.15678/) (2023)
- [Cross-Domain Video Anomaly Detection without Target Domain Adaptation](https://mlsanity.com/paper/2212.07010/) (2022)
- [Multi-Label Self-Supervised Learning with Scene Images](https://mlsanity.com/paper/2308.03286/) (2023)
- [Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigation](https://mlsanity.com/paper/2302.01520/) (2023)
- [Towards Zero-shot 3D Anomaly Localization](https://mlsanity.com/paper/2412.04304/) (2024)
- [Unveiling the Unseen: A Comprehensive Survey on Explainable Anomaly Detection in Images and Videos](https://mlsanity.com/paper/2302.06670/) (2023)
- [Coarse Is Better? A New Pipeline Towards Self-Supervised Learning with Uncurated Images](https://mlsanity.com/paper/2306.04244/) (2023)
- [Unsupervised 3D registration through optimization-guided cyclical self-training](https://mlsanity.com/paper/2306.16997/) (2023)
- [Vision Transformers in Precision Agriculture: A Comprehensive Survey](https://mlsanity.com/paper/2504.21706/) (2025)
- [Harnessing Synthetic Datasets: The Role of Shape Bias in Deep Neural Network Generalization](https://mlsanity.com/paper/2311.06224/) (2023)
- [Towards Long-Range ENSO Prediction with an Explainable Deep Learning Model](https://mlsanity.com/paper/2503.19502/) (2025)
- [Generative Poisoning Using Random Discriminators](https://mlsanity.com/paper/2211.01086/) (2022)
- [Revisiting Out-of-Distribution Detection in Real-time Object Detection: From Benchmark Pitfalls to a New Mitigation Paradigm](https://mlsanity.com/paper/2503.07330/) (2025)
- [Activation Subspaces for Out-of-Distribution Detection](https://mlsanity.com/paper/2508.21695/) (2025)
- [Understanding Gene Expression-to-Image Learning: Spatial Structure and Information Fidelity](https://mlsanity.com/paper/2610.17699/) (2026)

## Abstract (arXiv)

This paper starts by revealing a surprising finding: without any learning, a randomly initialized CNN can localize objects surprisingly well. That is, a CNN has an inductive bias to naturally focus on objects, named as Tobias ("The object is at sight") in this paper. This empirical inductive bias is further analyzed and successfully applied to self-supervised learning (SSL). A CNN is encouraged to learn representations that focus on the foreground object, by transforming every image into various versions with different backgrounds, where the foreground and background separation is guided by Tobias. Experimental results show that the proposed Tobias significantly improves downstream tasks, especially for object detection. This paper also shows that Tobias has consistent improvements on training sets of different sizes, and is more resilient to changes in image augmentations. Code is available at https://github.com/CupidJay/Tobias.