mlsanityData to 14 Oct 2026

Sanity checks for patch visualisation in prototype-based image classification

arXiv 2311.16120 ↗First posted 25 Oct 2023cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

In this work, we perform an analysis of the visualisation methods implemented in ProtoPNet and ProtoTree, two self-explaining visual classifiers based on prototypes. We show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour, which can create a false sense of bias in the model. We also demonstrate quantitatively that this issue can be mitigated by using other saliency methods that provide more faithful image patches.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

27 of 35 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  5. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  6. 2018Sanity Checks for Saliency Mapscited by 1,156
  7. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  8. 2014Striving for Simplicity: The All Convolutional Netcited by 990
  9. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  10. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  11. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  12. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
15 more
  1. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 297
  2. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 214
  3. 2016Grad-CAM: Why did you say that?cited by 161
  4. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  5. 2015A Large-Scale Car Dataset for Fine-Grained Categorization and Verificationcited by 130
  6. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  7. 2019Sanity Checks for Saliency Metricscited by 119
  8. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  9. 2020ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discoverycited by 89
  10. 2021This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networkscited by 65
  11. 2017Towards better understanding of gradient-based attribution methods for Deep Neural Networkscited by 54
  12. 2020This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognitioncited by 50
  13. 2021This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagationcited by 43
  14. 2022Metrics for saliency map evaluation of deep learning explanation methodscited by 40
  15. 2021Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AIcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 4.

8 papers.

  1. 2024LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Visioncited by 15
  2. 2024This actually looks like that: Proto-BagNets for local and global interpretability-by-designcited by 7
  3. 2024Prototype-based Interpretable Breast Cancer Prediction Models: Analysis and Challengescited by 3
  4. 2024CaBRNet, an open-source library for developing and evaluating Case-Based Reasoning Modelscited by 3
  5. 2025This looks like what? Challenges and Future Research Directions for Part-Prototype Modelscited by 3
  6. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
  7. 2025Comprehensive Evaluation of Prototype Neural Networkscited by 1
  8. 2025FaCT: Faithful Concept Traces for Explaining Neural Network Decisionscited by 1