# Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations

arXiv: https://arxiv.org/abs/2308.08162 · HTML: https://mlsanity.com/paper/2308.08162/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 65 references that are in the map.
- First posted on arXiv 16 Aug 2023.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Understanding Deep Networks via Extremal Perturbations and Smooth Masks](https://mlsanity.com/paper/1910.08485/) (2019)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Concept Whitening for Interpretable Image Recognition](https://mlsanity.com/paper/2002.01650/) (2020)
- [Neural Prototype Trees for Interpretable Fine-grained Image Recognition](https://mlsanity.com/paper/2012.02046/) (2020)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Attribute Prototype Network for Zero-Shot Learning](https://mlsanity.com/paper/2008.08290/) (2020)
- [Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded](https://mlsanity.com/paper/1902.03751/) (2019)
- [Technical Report on the CleverHans v2.1.0 Adversarial Examples Library](https://mlsanity.com/paper/1610.00768/) (2016)
- [Interpretable Image Classification with Differentiable Prototypes Assignment](https://mlsanity.com/paper/2112.02902/) (2021)
- [ProtGNN: Towards Self-Explaining Graph Neural Networks](https://mlsanity.com/paper/2112.00911/) (2021)
- [Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition](https://mlsanity.com/paper/1903.06150/) (2019)
- [ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery](https://mlsanity.com/paper/2011.14340/) (2020)
- [On the Connection Between Adversarial Robustness and Saliency Map Interpretability](https://mlsanity.com/paper/1905.04172/) (2019)
- [Interpreting Adversarially Trained Convolutional Neural Networks](https://mlsanity.com/paper/1905.09797/) (2019)
- [XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanations](https://mlsanity.com/paper/2103.10663/) (2021)
- [IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammography](https://mlsanity.com/paper/2103.12308/) (2021)
- [Interpretable Image Recognition with Hierarchical Prototypes](https://mlsanity.com/paper/1906.10651/) (2019)
- [This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networks](https://mlsanity.com/paper/2105.02968/) (2021)

## Built on by (most cited first, 27 of 27)

- [Interpretable Image Classification with Adaptive Prototype-based Vision Transformers](https://mlsanity.com/paper/2410.20722/) (2024)
- [Pixel-Grounded Prototypical Part Networks](https://mlsanity.com/paper/2309.14531/) (2023)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Vision](https://mlsanity.com/paper/2405.14331/) (2024)
- [ProtoS-ViT: Visual foundation models for sparse self-explainable classifications](https://mlsanity.com/paper/2406.10025/) (2024)
- [An Overview of Prototype Formulations for Interpretable Deep Learning](https://mlsanity.com/paper/2410.08925/) (2024)
- [Formal Abductive Latent Explanations for Prototype-Based Networks](https://mlsanity.com/paper/2511.16588/) (2025)
- [SIDE: Sparse Information Disentanglement for Explainable Artificial Intelligence](https://mlsanity.com/paper/2507.19321/) (2025)
- [Prototype-based Interpretable Breast Cancer Prediction Models: Analysis and Challenges](https://mlsanity.com/paper/2403.20260/) (2024)
- [This Probably Looks Exactly Like That: An Invertible Prototypical Network](https://mlsanity.com/paper/2407.12200/) (2024)
- [CaBRNet, an open-source library for developing and evaluating Case-Based Reasoning Models](https://mlsanity.com/paper/2409.16693/) (2024)
- [This looks like what? Challenges and Future Research Directions for Part-Prototype Models](https://mlsanity.com/paper/2502.09340/) (2025)
- [Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning](https://mlsanity.com/paper/2503.08636/) (2025)
- [Position: Do Not Explain Vision Models Without Context](https://mlsanity.com/paper/2404.18316/) (2024)
- [Revisiting FunnyBirds evaluation framework for prototypical parts networks](https://mlsanity.com/paper/2408.11401/) (2024)
- [Tell me why: Visual foundation models as self-explainable classifiers](https://mlsanity.com/paper/2502.19577/) (2025)
- [Comprehensive Evaluation of Prototype Neural Networks](https://mlsanity.com/paper/2507.06819/) (2025)
- [FaCT: Faithful Concept Traces for Explaining Neural Network Decisions](https://mlsanity.com/paper/2510.25512/) (2025)
- [SIC: Similarity-Based Interpretable Image Classification with Neural Networks](https://mlsanity.com/paper/2501.17328/) (2025)
- [Adversarial Training Improves Generalization Under Distribution Shifts in Bird Sound Classification](https://mlsanity.com/paper/2507.13727/) (2025)
- [Deeply Dual Supervised learning for melanoma recognition](https://mlsanity.com/paper/2508.01994/) (2025)
- [Edge Detection for Organ Boundaries via Top Down Refinement and SubPixel Upsampling](https://mlsanity.com/paper/2508.06805/) (2025)
- [DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentation](https://mlsanity.com/paper/2508.06816/) (2025)
- [VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagation](https://mlsanity.com/paper/2508.06819/) (2025)
- [ProtoQuant: Quantization of Prototypical Parts For General and Fine-Grained Image Classification](https://mlsanity.com/paper/2602.06592/) (2026)
- [Can Large Language Models Make Everyone Happy?](https://mlsanity.com/paper/2602.11091/) (2026)
- [Beyond Points: Spherical Distributional Part Prototypes for Interpretable Classification](https://mlsanity.com/paper/2606.27582/) (2026)

## Abstract (arXiv)

Prototypical parts-based networks are becoming increasingly popular due to their faithful self-explanations. However, their similarity maps are calculated in the penultimate network layer. Therefore, the receptive field of the prototype activation region often depends on parts of the image outside this region, which can lead to misleading interpretations. We name this undesired behavior a spatial explanation misalignment and introduce an interpretability benchmark with a set of dedicated metrics for quantifying this phenomenon. In addition, we propose a method for misalignment compensation and apply it to existing state-of-the-art models. We show the expressiveness of our benchmark and the effectiveness of the proposed compensation methodology through extensive empirical studies.