mlsanityData to 14 Oct 2026

This Looks Like Those: Illuminating Prototypical Concepts Using Multiple Visualizations

arXiv 2310.18589 ↗First posted 28 Oct 2023cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

We present ProtoConcepts, a method for interpretable image classification combining deep learning and case-based reasoning using prototypical parts. Existing work in prototype-based image classification uses a ``this looks like that'' reasoning process, which dissects a test image by finding prototypical parts and combining evidence from these prototypes to make a final classification. However, all of the existing prototypical part-based image classifiers provide only one-to-one comparisons, where a single training image patch serves as a prototype to compare with a part of our test image. With these single-image comparisons, it can often be difficult to identify the underlying concept being compared (e.g., ``is it comparing the color or the shape?''). Our proposed method modifies the architecture of prototype-based networks to instead learn prototypical concepts which are visualized using multiple image patches. Having multiple visualizations of the same prototype allows us to more easily identify the concept captured by that prototype (e.g., ``the test image and the related training patches are all the same shade of blue''), and allows our model to create richer, more interpretable visual explanations. Our experiments show that our ``this looks like those'' reasoning process can be applied as a modification to a wide range of existing prototypical image classification networks while achieving comparable accuracy on benchmark datasets.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 28 Oct 2023.

Builds on

26 of 40 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  8. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  9. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  10. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  11. 2018Sanity Checks for Saliency Mapscited by 1,156
  12. 2017SmoothGrad: removing noise by adding noisecited by 1,109
14 more
  1. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  2. 2015Understanding Neural Networks Through Deep Visualizationcited by 334
  3. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
  4. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  5. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 214
  6. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  7. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  8. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  9. 2019The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanationscited by 97
  10. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  11. 2020ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discoverycited by 89
  12. 2021IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammographycited by 72
  13. 2020This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognitioncited by 50
  14. 2016Detecting Unexpected Obstacles for Self-Driving Cars: Fusing Deep Learning and Geometric Modelingcited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 15; 2026: 5.

36 papers.

Sort:

By area:

  1. 2024Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretationscited by 33
  2. 2024Interpretable Image Classification with Adaptive Prototype-based Vision Transformerscited by 25
  3. 2024Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlookscited by 18
  4. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  5. 2024LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Visioncited by 15
  6. 2025Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Timecited by 13
  7. 2025Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysiscited by 12
  8. 2023Mixture of Gaussian-distributed Prototypes with Generative Modelling for Interpretable and Trustworthy Image Recognitioncited by 11
  9. 2024This Looks Better than That: Better Interpretable Models with ProtoPNeXtcited by 11
  10. 2024AudioProtoPNet: An interpretable deep learning model for bird sound classificationcited by 10
  11. 2024ProtoS-ViT: Visual foundation models for sparse self-explainable classificationscited by 8
  12. 2024Post-hoc Part-prototype Networkscited by 7
24 more
  1. 2025Geodesic Prototype Matching via Diffusion Maps for Interpretable Fine-Grained Recognitioncited by 6
  2. 2024An Overview of Prototype Formulations for Interpretable Deep Learningcited by 5
  3. 2025ProtoECGNet: Case-Based Interpretable Deep Learning for Multi-Label ECG Classification with Contrastive Learningcited by 4
  4. 2025Locality-aware Concept Bottleneck Modelcited by 4
  5. 2024This Probably Looks Exactly Like That: An Invertible Prototypical Networkcited by 3
  6. 2024Achieving Domain-Independent Certified Robustness via Knowledge Continuitycited by 3
  7. 2025This looks like what? Challenges and Future Research Directions for Part-Prototype Modelscited by 3
  8. 2025Personalized Interpretability -- Interactive Alignment of Prototypical Parts Networkscited by 3
  9. 2025Exploring Interpretability for Visual Prompt Tuning with Cross-layer Conceptscited by 2
  10. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  11. 2026Prototype-Grounded Concept Models for Verifiable Concept Alignmentcited by 2
  12. 2024FPN-IAIA-BL: A Multi-Scale Interpretable Deep Learning Model for Classification of Mass Margins in Digital Mammographycited by 1
  13. 2024Exploiting Interpretable Capabilities with Concept-Enhanced Diffusion and Prototype Networkscited by 1
  14. 2025COMIX: Compositional Explanations using Prototypescited by 1
  15. 2025Comprehensive Evaluation of Prototype Neural Networkscited by 1
  16. 2024What Do You See in Common? Learning Hierarchical Prototypes over Tree-of-Life to Discover Evolutionary Traitscited by 0
  17. 2024Bi-ICE: An Inner Interpretable Framework for Image Classification via Bi-directional Interactions between Concept and Input Embeddingscited by 0
  18. 2025Enhancing Job Salary Prediction with Disentangled Composition Effect Modeling: A Neural Prototyping Approachcited by 0
  19. 2025ProtoVQA: An Adaptable Prototypical Framework for Explainable Fine-Grained Visual Question Answeringcited by 0
  20. 2025What You See is (Usually) What You Get: Multimodal Prototype Networks that Abstain from Expensive Modalitiescited by 0
  21. 2026Concepts from Representations: Post-hoc Concept Bottleneck Models via Sparse Decomposition of Visual Representationscited by 0
  22. 2026Beyond Points: Spherical Distributional Part Prototypes for Interpretable Classificationcited by 0
  23. 2026GRAPE: Graph-Augmented Prototype Explanations for Interactive Medical Image Diagnosiscited by 0
  24. 2026Diverse by Design: Architectural Constraints for Prototype-Based Interpretabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Interpretable Image Classification via Non-parametric Part Prototype Learning
  2. 2022Towards Human-Interpretable Prototypes for Visual Assessment of Image Classification Models
  3. 2026ProtoQuant: Quantization of Prototypical Parts For General and Fine-Grained Image Classification
  4. 2023Improving Prototypical Visual Explanations with Reward Reweighing, Reselection, and Retraining
  5. 2025Semantically Guided Dynamic Visual Prototype Refinement for Compositional Zero-Shot Learning
  6. 2023Seeing the Intangible: Survey of Image Classification into High-Level and Abstract Categories
  7. 2025Dynamic Multimodal Prototype Learning in Vision-Language Models
  8. 2026Symbolic Grounding Reveals Representational Bottlenecks in Abstract Visual Reasoning