Object Detectors Emerge in Deep Scene CNNs
Not on the map. Why?
Abstract (arXiv)
With the success of new computational architectures for visual processing, such as convolutional neural networks (CNN) and access to image databases with millions of labeled examples (e.g., ImageNet, Places), the state of the art in computer vision is advancing rapidly. One important factor for continued progress is to understand the representations that are learned by the inner layers of these deep architectures. Here we show that object detectors emerge from training CNNs to perform scene classification. As scenes are composed of objects, the CNN for scene classification automatically discovers meaningful objects detectors, representative of the learned scene categories. With object detectors emerging as a result of learning to recognize scenes, our work demonstrates that the same network can perform both scene recognition and object localization in a single forward-pass, without ever having been explicitly taught the notion of objects.
Key facts. Cited by 178 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Dec 2014.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 56; 2022: 39; 2023: 33; 2024: 27; 2025: 17; 2026: 6.
178 papers.
Sort:
By area:
- 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
- 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
- 2021Anabranch Network for Camouflaged Object Segmentationcited by 227
- 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
- 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 144
- 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
- 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
- 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 97
- 2021An Interpretability Illusion for BERTcited by 94
- 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
- 2022Sparse Mixture-of-Experts are Domain Generalizable Learnerscited by 85
- 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
38 more
- 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
- 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
- 2022On the Symmetries of Deep Learning Models and their Internal Representationscited by 63
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
- 2021Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanationscited by 56
- 2023Learning Bottleneck Concepts in Image Classificationcited by 49
- 2022Modeling Image Composition for Complex Scene Generationcited by 43
- 2021Context-Aware Layout to Image Generation with Enhanced Object Appearancecited by 41
- 2021STEEX: Steering Counterfactual Explanations with Semanticscited by 40
- 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
- 2021EigenGAN: Layer-Wise Eigen-Learning for GANscited by 34
- 2022Explaining Deepfake Detection by Analysing Image Matchingcited by 34
- 2023LINe: Out-of-Distribution Detection by Leveraging Important Neuronscited by 34
- 2024Gradient based Feature Attribution in Explainable AI: A Technical Reviewcited by 34
- 2022Interpreting Neural Networks through the Polytope Lenscited by 33
- 2023A Preliminary Study of the Intrinsic Relationship between Complexity and Alignmentcited by 33
- 2021Neural Response Interpretation through the Lens of Critical Pathwayscited by 32
- 2023FIND: A Function Description Benchmark for Evaluating Interpretability Methodscited by 32
- 2021Space-Time Crop & Attend: Improving Cross-modal Video Representation Learningcited by 31
- 2022OCTET: Object-aware Counterfactual Explanationscited by 28
- 2021Looking Outside the Window: Wide-Context Transformer for the Semantic Segmentation of High-Resolution Remote Sensing Imagescited by 27
- 2023Does a Neural Network Really Encode Symbolic Concepts?cited by 27
- 2023Circuit Breaking: Removing Model Behaviors with Targeted Ablationcited by 27
- 2021Finding Discriminative Filters for Specific Degradations in Blind Super-Resolutioncited by 26
- 2024Large Multi-modal Models Can Interpret Features in Large Multi-modal Modelscited by 23
- 2021Carrying out CNN Channel Pruning in a White Boxcited by 22
- 2024Optimal ablation for interpretabilitycited by 21
- 2022Channel Importance Matters in Few-Shot Image Classificationcited by 20
- 2022Contrastive Deep Supervisioncited by 20
- 2024Linear Explanations for Individual Neuronscited by 19
- 2022HINT: Hierarchical Neuron Concept Explainercited by 17
- 2022Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanationscited by 17
- 2024CausalFormer: An Interpretable Transformer for Temporal Causal Discoverycited by 17
- 2021Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoringcited by 16
- 2023Soft Matching Distance: A metric on neural representations that captures single-neuron tuningcited by 16
- 2021Detection and Localization of Facial Expression Manipulationscited by 15
- 2021Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Posecited by 15
- 2023Towards a fuller understanding of neurons with Clustered Compositional Explanationscited by 15
50 most cited of 178 All 178 (JSON)
- 2026Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics3 Jun 2026
- 2026Explainable Part-Based Vehicle Classifier with Spatial Awareness8 May 2026
- 2026Self-Distillation of Hidden Layers for Self-Supervised Representation Learning16 Mar 2026
- 2026HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token5 Mar 2026
- 2026T1: One-to-One Channel-Head Binding for Multivariate Time-Series Imputation24 Feb 2026
- 2026ObjectVisA-120: Object-based Visual Attention Prediction in Interactive Street-crossing Environments19 Jan 2026
- 2025Faithful and Stable Neuron Explanations for Trustworthy Mechanistic Interpretability19 Dec 2025
- 2025Probing the Probes: Methods and Metrics for Concept Alignment6 Nov 2025
- 2025Token-Based Detection of Spurious Correlations in Vision Transformers4 Sep 2025
- 2025Superposition in Graph Neural Networks31 Aug 2025
- 2025Model Interpretability and Rationale Extraction by Input Mask Optimization15 Aug 2025
- 2025Tobler's First Law in GeoAI: A Spatially Explicit Deep Learning Model for Terrain Feature Detection Under Weak Supervision1 Aug 2025
38 more
- 2025Towards White-Box Deep Wireless Sensing29 Jul 2025
- 2025LMPNet for Weakly-supervised Keypoint Discovery3 Jul 2025
- 2025When concept-based XAI is imprecise: Do people distinguish between generalisations and misrepresentations?22 Jun 2025
- 2025Evaluating Neuron Explanations: A Unified Framework with Sanity Checks6 Jun 2025
- 2025The Spotlight Resonance Method: Resolving the Alignment of Embedded Activations9 May 2025
- 2025VLAs are Confined yet Capable of Generalizing to Novel Instructions6 May 2025
- 2025Measuring the (Un)Faithfulness of Concept-Based Explanations15 Apr 2025
- 2025Towards Human-Understandable Multi-Dimensional Concept Discovery24 Mar 2025
- 2025Superscopes: Amplifying Internal Feature Representations for Language Model Interpretation3 Mar 2025
- 2025Faithful Counterfactual Visual Explanations (FCVE)12 Jan 2025
- 2025Explainable Neural Networks with Guarantees: A Sparse Estimation Approach2 Jan 2025
- 2024Large Multi-modal Models Can Interpret Features in Large Multi-modal Models22 Nov 2024
- 2024LightFFDNets: Lightweight Convolutional Neural Networks for Rapid Facial Forgery Detection18 Nov 2024
- 2024Choosing the right basis for interpretability: Psychophysical comparison between neuron-based and dictionary-based representations6 Nov 2024
- 2024Probing Ranking LLMs: A Mechanistic Analysis for Information Retrieval24 Oct 2024
- 2024Emergent Visual Grounding in Large Multimodal Models Without Grounding Supervision10 Oct 2024
- 2024Linking in Style: Understanding learned features in deep learning models25 Sep 2024
- 2024Optimal ablation for interpretability16 Sep 2024
- 2024Quantum Gradient Class Activation Map for Model Interpretability12 Aug 2024
- 2024Machine Learning Applications in Medical Prognostics: A Comprehensive Review5 Aug 2024
- 2024Machine learning surrogates for efficient hydrologic modeling: Insights from stochastic simulations of managed aquifer recharge30 Jul 2024
- 2024LaFAM: Unsupervised Feature Attribution with Label-free Activation Maps8 Jul 2024
- 2024CausalFormer: An Interpretable Transformer for Temporal Causal Discovery24 Jun 2024
- 2024A Neurosymbolic Framework for Bias Correction in Convolutional Neural Networks24 May 2024
- 2024Linear Explanations for Individual Neurons10 May 2024
- 2024A Primer on the Inner Workings of Transformer-based Language Models30 Apr 2024
- 2024CA-Stream: Attention-based pooling for interpretable image recognition23 Apr 2024
- 2024Automatic Discovery of Visual Circuits22 Apr 2024
- 2024Toward a Realistic Benchmark for Out-of-Distribution Detection16 Apr 2024
- 2024Concept-Attention Whitening for Interpretable Skin Lesion Diagnosis9 Apr 2024
- 2024MCNet: A crowd denstity estimation network based on integrating multiscale attention module29 Mar 2024
- 2024Interpreting Neurons in Deep Vision Networks with Language Models20 Mar 2024
- 2024XPose: eXplainable Human Pose Estimation19 Mar 2024
- 2024Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Driving18 Mar 2024
- 2024Gradient based Feature Attribution in Explainable AI: A Technical Review15 Mar 2024
- 2024HOLMES: HOLonym-MEronym based Semantic inspection for Convolutional Image Classifiers13 Mar 2024
- 2024Feature CAM: Interpretable AI in Image Classification8 Mar 2024
- 2024On The Potential of The Fractal Geometry and The CNNs Ability to Encode it7 Jan 2024
50 newest of 178 All 178 (JSON)
- 2024Large Multi-modal Models Can Interpret Features in Large Multi-modal Modelscited by 23
- 2023Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Modelscited by 15
- 2026HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Tokencited by 13
- 2024Emergent Visual Grounding in Large Multimodal Models Without Grounding Supervisioncited by 8
- 2026HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token5 Mar 2026
- 2024Large Multi-modal Models Can Interpret Features in Large Multi-modal Models22 Nov 2024
- 2024Emergent Visual Grounding in Large Multimodal Models Without Grounding Supervision10 Oct 2024
- 2023Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models28 Nov 2023
- 2022Sparse Mixture-of-Experts are Domain Generalizable Learnerscited by 85
- 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
- 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
- 2021Anabranch Network for Camouflaged Object Segmentationcited by 227
- 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
- 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 144
- 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
- 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
- 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 97
- 2021An Interpretability Illusion for BERTcited by 94
- 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
- 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
- 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
12 most cited of 170 All 170 (JSON)
- 2026Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics3 Jun 2026
- 2026Explainable Part-Based Vehicle Classifier with Spatial Awareness8 May 2026
- 2026Self-Distillation of Hidden Layers for Self-Supervised Representation Learning16 Mar 2026
- 2026T1: One-to-One Channel-Head Binding for Multivariate Time-Series Imputation24 Feb 2026
- 2026ObjectVisA-120: Object-based Visual Attention Prediction in Interactive Street-crossing Environments19 Jan 2026
- 2025Faithful and Stable Neuron Explanations for Trustworthy Mechanistic Interpretability19 Dec 2025
- 2025Probing the Probes: Methods and Metrics for Concept Alignment6 Nov 2025
- 2025Token-Based Detection of Spurious Correlations in Vision Transformers4 Sep 2025
- 2025Superposition in Graph Neural Networks31 Aug 2025
- 2025Model Interpretability and Rationale Extraction by Input Mask Optimization15 Aug 2025
- 2025Tobler's First Law in GeoAI: A Spatially Explicit Deep Learning Model for Terrain Feature Detection Under Weak Supervision1 Aug 2025
- 2025Towards White-Box Deep Wireless Sensing29 Jul 2025
12 newest of 170 All 170 (JSON)