The Overfocusing Bias of Convolutional Neural Networks: A Saliency-Guided Regularization Approach
Not on the map. Why?
Abstract (arXiv)
Despite transformers being considered as the new standard in computer vision, convolutional neural networks (CNNs) still outperform them in low-data regimes. Nonetheless, CNNs often make decisions based on narrow, specific regions of input images, especially when training data is limited. This behavior can severely compromise the model's generalization capabilities, making it disproportionately dependent on certain features that might not represent the broader context of images. While the conditions leading to this phenomenon remain elusive, the primary intent of this article is to shed light on this observed behavior of neural networks. Our research endeavors to prioritize comprehensive insight and to outline an initial response to this phenomenon. In line with this, we introduce Saliency Guided Dropout (SGDrop), a pioneering regularization approach tailored to address this specific issue. SGDrop utilizes attribution methods on the feature map to identify and then reduce the influence of the most salient features during training. This process encourages the network to diversify its attention and not focus solely on specific standout areas. Our experiments across several visual classification benchmarks validate SGDrop's role in enhancing generalization. Significantly, models incorporating SGDrop display more expansive attributions and neural activity, offering a more comprehensive view of input images in contrast to their traditionally trained counterparts.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 48 of its 57 references that are in the map. First posted on arXiv 25 Sep 2024.
Builds on
48 of 57 references.
References · 48
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2022A ConvNet for the 2020scited by 3,545
- 2017Axiomatic Attribution for Deep Networkscited by 3,350
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
- 2017SmoothGrad: removing noise by adding noisecited by 1,097
36 more
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
- 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
- 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
- 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
- 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
- 2016Top-down Neural Attention by Excitation Backpropcited by 318
- 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 307
- 2022How Do Vision Transformers Work?cited by 305
- 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 293
- 2018DropBlock: A regularization method for convolutional networkscited by 261
- 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
- 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 260
- 2014Efficient Object Localization Using Convolutional Networkscited by 226
- 2017Exploring Models and Data for Remote Sensing Image Caption Generationcited by 195
- 2020SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularizationcited by 149
- 2021Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversitycited by 127
- 2019Attention-based Dropout Layer for Weakly Supervised Object Localizationcited by 125
- 2022What to Hide from Your Students: Attention-Guided Masked Image Modelingcited by 117
- 2021Efficient Training of Visual Transformers with Small Datasetscited by 107
- 2017PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrievalcited by 96
- 2021Improving Deep Learning Interpretability by Saliency Guided Trainingcited by 68
- 2021Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysiscited by 65
- 2022Harmonizing the object recognition strategies of deep neural networks with humanscited by 59
- 2020KeepAugment: A Simple Information-Preserving Data Augmentation Approachcited by 52
- 2017Curriculum Dropoutcited by 50
- 2021AutoDropout: Learning Dropout Patterns to Regularize Deep Networkscited by 35
- 2022Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measurecited by 33
- 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
- 2022Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasetscited by 28
- 2023Opti-CAM: Optimizing saliency maps for interpretabilitycited by 25
- 2019Saliency Learning: Teaching the Model Where to Pay Attentioncited by 23
- 2023GuidedMixup: An Efficient Mixup Strategy Guided by Saliency Mapscited by 23
- 2022Local Feature Swapping for Generalization in Reinforcement Learningcited by 9
- 2018Excitation Dropout: Encouraging Plasticity in Deep Neural Networkscited by 7
- 2020MaxDropout: Deep Neural Network Regularization Based on Maximum Output Valuescited by 5
- 2022AD-DROP: Attribution-Driven Dropout for Robust Language Model Fine-Tuningcited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.