mlsanityData to 7 Oct 2026

CAM-Based Methods Can See through Walls

arXiv 2404.01964 ↗First posted 2 Apr 2024cs.CV cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

CAM-based methods are widely-used post-hoc interpretability method that produce a saliency map to explain the decision of an image classification model. The saliency map highlights the important areas of the image relevant to the prediction. In this paper, we show that most of these methods can incorrectly attribute an important score to parts of the image that the model cannot see. We show that this phenomenon occurs both theoretically and experimentally. On the theory side, we analyze the behavior of GradCAM on a simple masked CNN model at initialization. Experimentally, we train a VGG-like model constrained to not use the lower part of the image and nevertheless observe positive scores in the unseen part of the image. This behavior is evaluated quantitatively on two new datasets. We believe that this is problematic, potentially leading to mis-interpretation of the model's behavior.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

25 of 38 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  8. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  9. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  11. 2018Sanity Checks for Saliency Mapscited by 1,149
  12. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
13 more
  1. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  2. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  3. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  4. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  5. 2017Interpretation of Neural Networks is Fragilecited by 443
  6. 2017The (Un)reliability of saliency methodscited by 331
  7. 2020A Survey on Neural Network Interpretabilitycited by 291
  8. 2020Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNscited by 151
  9. 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
  10. 2018On the Convergence Rate of Training Recurrent Neural Networkscited by 82
  11. 2020Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networkscited by 79
  12. 2023Opti-CAM: Optimizing saliency maps for interpretabilitycited by 25
  13. 2021What does LIME really see in images?cited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024Linearly-Interpretable Concept Embedding Models for Text Analysiscited by 3
  2. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
  3. 2026Attribution Upsampling should Redistribute, Not Interpolatecited by 0
  4. 2026Evidential Rule Learning for Interpretable Classification with Abstentioncited by 0