mlsanityData to 7 Oct 2026

A Random CNN Sees Objects: One Inductive Bias of CNN and Its Applications

arXiv 2106.09259 ↗First posted 17 Jun 2021cs.CV cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

This paper starts by revealing a surprising finding: without any learning, a randomly initialized CNN can localize objects surprisingly well. That is, a CNN has an inductive bias to naturally focus on objects, named as Tobias ("The object is at sight") in this paper. This empirical inductive bias is further analyzed and successfully applied to self-supervised learning (SSL). A CNN is encouraged to learn representations that focus on the foreground object, by transforming every image into various versions with different backgrounds, where the foreground and background separation is guided by Tobias. Experimental results show that the proposed Tobias significantly improves downstream tasks, especially for object detection. This paper also shows that Tobias has consistent improvements on training sets of different sizes, and is more resilient to changes in image augmentations. Code is available at https://github.com/CupidJay/Tobias.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

34 of 42 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  11. 2017Mask R-CNNcited by 7,259
  12. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
22 more
  1. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  7. 2020Supervised Contrastive Learningcited by 2,803
  8. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  11. 2017Deep Image Priorcited by 1,097
  12. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  13. 2017Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detectioncited by 209
  14. 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 200
  15. 2020The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Modelscited by 91
  16. 2020Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learningcited by 62
  17. 2016Inductive Bias of Deep Convolutional Networks through Pooling Geometrycited by 53
  18. 2020Rethinking the Route Towards Weakly Supervised Object Localizationcited by 46
  19. 2016Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrievalcited by 44
  20. 2020The Lottery Ticket Hypothesis for Object Recognitioncited by 34
  21. 2017Unsupervised Object Discovery and Co-Localization by Deep Descriptor Transformingcited by 15
  22. 2021Rethinking Self-Supervised Learning: Small is Beautifulcited by 13

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 7; 2024: 1; 2025: 4.

14 papers.

Sort:

By area:

  1. 2023DisWOT: Student Architecture Search for Distillation WithOut Trainingcited by 29
  2. 2022Cross-Domain Video Anomaly Detection without Target Domain Adaptationcited by 20
  3. 2023Multi-Label Self-Supervised Learning with Scene Imagescited by 17
  4. 2023Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigationcited by 8
  5. 2024Towards Zero-shot 3D Anomaly Localizationcited by 8
  6. 2023Unveiling the Unseen: A Comprehensive Survey on Explainable Anomaly Detection in Images and Videoscited by 7
  7. 2023Unsupervised 3D registration through optimization-guided cyclical self-trainingcited by 7
  8. 2025Vision Transformers in Precision Agriculture: A Comprehensive Surveycited by 6
  9. 2023Coarse Is Better? A New Pipeline Towards Self-Supervised Learning with Uncurated Imagescited by 5
  10. 2023Harnessing Synthetic Datasets: The Role of Shape Bias in Deep Neural Network Generalizationcited by 4
  11. 2022Generative Poisoning Using Random Discriminatorscited by 3
  12. 2025Towards Long-Range ENSO Prediction with an Explainable Deep Learning Modelcited by 3
2 more
  1. 2025Revisiting Out-of-Distribution Detection in Real-time Object Detection: From Benchmark Pitfalls to a New Mitigation Paradigmcited by 2
  2. 2025Activation Subspaces for Out-of-Distribution Detectioncited by 2