Inverting and Understanding Object Detectors
Not on the map. Why?
Abstract (arXiv)
As a core problem in computer vision, the performance of object detection has improved drastically in the past few years. Despite their impressive performance, object detectors suffer from a lack of interpretability. Visualization techniques have been developed and widely applied to introspect the decisions made by other kinds of deep learning models; however, visualizing object detectors has been underexplored. In this paper, we propose using inversion as a primary tool to understand modern object detectors and develop an optimization-based approach to layout inversion, allowing us to generate synthetic images recognized by trained detectors as containing a desired configuration of objects. We reveal intriguing properties of detectors by applying our layout inversion technique to a variety of modern object detectors, and further investigate them via validation experiments: they rely on qualitatively different features for classification and regression; they learn canonical motifs of commonly co-occurring objects; they use diff erent visual cues to recognize objects of varying sizes. We hope our insights can help practitioners improve object detectors.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 58 of its 71 references that are in the map. First posted on arXiv 26 Jun 2021.
Builds on
58 of 71 references.
References · 58
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2020End-to-End Object Detection with Transformerscited by 7,384
- 2017Mask R-CNNcited by 7,259
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2016Feature Pyramid Networks for Object Detectioncited by 5,021
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2015SSD: Single Shot MultiBox Detectorcited by 3,125
- 2015Fast R-CNNcited by 2,930
- 2018YOLOv3: An Incremental Improvementcited by 2,652
- 2015Learning Deep Features for Discriminative Localizationcited by 2,430
38 more
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
- 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
- 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
- 2016YOLO9000: Better, Faster, Strongercited by 1,429
- 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
- 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
- 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
- 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
- 2019Objects as Pointscited by 1,172
- 2017SmoothGrad: removing noise by adding noisecited by 1,097
- 2014Striving for Simplicity: The All Convolutional Netcited by 983
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
- 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
- 2017Focal Loss for Dense Object Detectioncited by 749
- 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
- 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
- 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
- 2014Understanding Deep Image Representations by Inverting Themcited by 399
- 2015Understanding Neural Networks Through Deep Visualizationcited by 332
- 2018Image Generation from Scene Graphscited by 324
- 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
- 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
- 2015Visualizing and Understanding Neural Models in NLPcited by 199
- 2016Speed/accuracy trade-offs for modern convolutional object detectorscited by 198
- 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
- 2014Object Detectors Emerge in Deep Scene CNNscited by 178
- 2018Acquisition of Localization Confidence for Accurate Object Detectioncited by 174
- 2015Inverting Visual Representations with Convolutional Networkscited by 168
- 2014Simultaneous Detection and Segmentationcited by 158
- 2015Visualizing and Understanding Recurrent Networkscited by 158
- 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
- 2016Learning What and Where to Drawcited by 105
- 2018Image Generation from Layoutcited by 105
50 most cited of 58 All 58 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.