mlsanityData to 7 Oct 2026

Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations

arXiv 2308.08162 ↗First posted 16 Aug 2023cs.CV cs.AI cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Prototypical parts-based networks are becoming increasingly popular due to their faithful self-explanations. However, their similarity maps are calculated in the penultimate network layer. Therefore, the receptive field of the prototype activation region often depends on parts of the image outside this region, which can lead to misleading interpretations. We name this undesired behavior a spatial explanation misalignment and introduce an interpretability benchmark with a set of dedicated metrics for quantifying this phenomenon. In addition, we propose a method for misalignment compensation and apply it to existing state-of-the-art models. We show the expressiveness of our benchmark and the effectiveness of the proposed compensation methodology through extensive empirical studies.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 41 of its 65 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

41 of 65 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  5. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  6. 2018Sanity Checks for Saliency Mapscited by 1,149
  7. 2018Robustness May Be at Odds with Accuracycited by 873
  8. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  9. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  10. 2017Adversarial Attacks on Neural Network Policiescited by 314
  11. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  12. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
29 more
  1. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
  2. 2017Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictionscited by 234
  3. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  4. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
  5. 2021Visual Saliency Transformercited by 175
  6. 2020Attribute Prototype Network for Zero-Shot Learningcited by 160
  7. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 127
  8. 2016Technical Report on the CleverHans v2.1.0 Adversarial Examples Librarycited by 123
  9. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  10. 2021ProtGNN: Towards Self-Explaining Graph Neural Networkscited by 96
  11. 2019Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognitioncited by 95
  12. 2020ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discoverycited by 89
  13. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  14. 2019Interpreting Adversarially Trained Convolutional Neural Networkscited by 75
  15. 2021XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanationscited by 72
  16. 2021IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammographycited by 72
  17. 2021This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networkscited by 65
  18. 2019Interpretable Image Recognition with Hierarchical Prototypescited by 64
  19. 2020There and Back Again: Revisiting Backpropagation Saliency Methodscited by 60
  20. 2020This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognitioncited by 50
  21. 2021This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagationcited by 43
  22. 2019A Survey of Black-Box Adversarial Attacks on Computer Vision Modelscited by 36
  23. 2022ProtoVAE: A Trustworthy Self-Explainable Prototypical Variational Modelcited by 34
  24. 2023ProtoSeg: Interpretable Semantic Segmentation with Prototypical Partscited by 33
  25. 2019Explaining Deep Classification of Time-Series Data with Learned Prototypescited by 30
  26. 2023ICICLE: Interpretable Class Incremental Continual Learningcited by 24
  27. 2021ProtoMIL: Multiple Instance Learning with Prototypical Parts for Whole-Slide Image Classificationcited by 22
  28. 2020Interpretable Deep Models for Cardiac Resynchronisation Therapy Response Predictioncited by 17
  29. 2022ProGReST: Prototypical Graph Regression Soft Trees for Molecular Property Predictioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 14; 2026: 3.

27 papers.

Sort:

By area:

  1. 2024Interpretable Image Classification with Adaptive Prototype-based Vision Transformerscited by 25
  2. 2023Pixel-Grounded Prototypical Part Networkscited by 18
  3. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  4. 2024LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Visioncited by 15
  5. 2024ProtoS-ViT: Visual foundation models for sparse self-explainable classificationscited by 8
  6. 2025Formal Abductive Latent Explanations for Prototype-Based Networkscited by 5
  7. 2024An Overview of Prototype Formulations for Interpretable Deep Learningcited by 4
  8. 2024Prototype-based Interpretable Breast Cancer Prediction Models: Analysis and Challengescited by 3
  9. 2024This Probably Looks Exactly Like That: An Invertible Prototypical Networkcited by 3
  10. 2024CaBRNet, an open-source library for developing and evaluating Case-Based Reasoning Modelscited by 3
  11. 2025This looks like what? Challenges and Future Research Directions for Part-Prototype Modelscited by 3
  12. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
15 more
  1. 2025SIDE: Sparse Information Disentanglement for Explainable Artificial Intelligencecited by 3
  2. 2024Position: Do Not Explain Vision Models Without Contextcited by 2
  3. 2024Revisiting FunnyBirds evaluation framework for prototypical parts networkscited by 2
  4. 2025Tell me why: Visual foundation models as self-explainable classifierscited by 2
  5. 2025Comprehensive Evaluation of Prototype Neural Networkscited by 1
  6. 2025FaCT: Faithful Concept Traces for Explaining Neural Network Decisionscited by 1
  7. 2025SIC: Similarity-Based Interpretable Image Classification with Neural Networkscited by 0
  8. 2025Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classificationcited by 0
  9. 2025Deeply Dual Supervised learning for melanoma recognitioncited by 0
  10. 2025Edge Detection for Organ Boundaries via Top Down Refinement and SubPixel Upsamplingcited by 0
  11. 2025DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentationcited by 0
  12. 2025VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagationcited by 0
  13. 2026ProtoQuant: Quantization of Prototypical Parts For General and Fine-Grained Image Classificationcited by 0
  14. 2026Can Large Language Models Make Everyone Happy?cited by 0
  15. 2026Beyond Points: Spherical Distributional Part Prototypes for Interpretable Classificationcited by 0