mlsanityData to 7 Oct 2026

ProtoMask: Segmentation-Guided Prototype Learning

arXiv 2510.00683 ↗First posted 1 Oct 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

XAI gained considerable importance in recent years. Methods based on prototypical case-based reasoning have shown a promising improvement in explainability. However, these methods typically rely on additional post-hoc saliency techniques to explain the semantics of learned prototypes. Multiple critiques have been raised about the reliability and quality of such techniques. For this reason, we study the use of prominent image segmentation foundation models to improve the truthfulness of the mapping between embedding and input space. We aim to restrict the computation area of the saliency map to a predefined semantic image patch to reduce the uncertainty of such visualizations. To perceive the information of an entire image, we use the bounding box from each generated segmentation mask to crop the image. Each mask results in an individual input in our novel model architecture named ProtoMask. We conduct experiments on three popular fine-grained classification datasets with a wide set of metrics, providing a detailed overview on explainability characteristics. The comparison with other popular models demonstrates competitive performance and unique explainability features of our model. https://github.com/uos-sis/quanproto

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

33 of 44 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Segment Anythingcited by 8,158
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  7. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  8. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  9. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  10. 2023Segment Everything Everywhere All at Oncecited by 546
  11. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  12. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 213
21 more
  1. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  2. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  3. 2020ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discoverycited by 89
  4. 2021This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networkscited by 65
  5. 2019Interpretable Image Recognition with Hierarchical Prototypescited by 64
  6. 2022ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognitioncited by 63
  7. 2021This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagationcited by 43
  8. 2023Learning Support and Trivial Prototypes for Interpretable Image Classificationcited by 38
  9. 2022Evaluation and Improvement of Interpretability for Self-Explainable Part-Prototype Networkscited by 37
  10. 2023Explain Any Concept: Segment Anything Meets Concept-Based Explanationcited by 37
  11. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  12. 2024Interpretable Image Classification with Adaptive Prototype-based Vision Transformerscited by 25
  13. 2024Subobject-level Image Tokenizationcited by 20
  14. 2023The Co-12 Recipe for Evaluating Interpretable Part-Prototype Image Classifierscited by 15
  15. 2024This Looks Better than That: Better Interpretable Models with ProtoPNeXtcited by 10
  16. 2024This actually looks like that: Proto-BagNets for local and global interpretability-by-designcited by 7
  17. 2024Vision Transformers with Natural Language Semanticscited by 5
  18. 2023Sanity checks and improvements for patch visualisation in prototype-based image classificationcited by 3
  19. 2024LCE: A Framework for Explainability of DNNs for Ultrasound Image Based on Concept Discoverycited by 2
  20. 2025FeatInv: Spatially resolved mapping from feature space to input space using conditional diffusion modelscited by 2
  21. 2025Comprehensive Evaluation of Prototype Neural Networkscited by 1

Built on by

Not cited yet.