mlsanityData to 14 Oct 2026

ProtoS-ViT: Visual foundation models for sparse self-explainable classifications

arXiv 2406.10025 ↗First posted 14 Jun 2024cs.CV cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Prototypical networks aim to build intrinsically explainable models based on the linear summation of concepts. Concepts are coherent entities that we, as humans, can recognize and associate with a certain object or entity. However, important challenges remain in the fair evaluation of explanation quality provided by these models. This work first proposes an extensive set of quantitative and qualitative metrics which allow to identify drawbacks in current prototypical networks. It then introduces a novel architecture which provides compact explanations, outperforming current prototypical models in terms of explanation quality. Overall, the proposed architecture demonstrates how frozen pre-trained ViT backbones can be effectively turned into prototypical models for both general and domain-specific tasks, in our case biomedical image classifiers. Code is available at urlhttps://github.com/hturbe/protosvit.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 14 Jun 2024.

Builds on

42 of 61 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  6. 2014Going Deeper with Convolutionscited by 4,241
  7. 2018Sanity Checks for Saliency Mapscited by 1,156
  8. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  9. 2023Vision Transformers Need Registerscited by 871
  10. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  11. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  12. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
30 more
  1. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  2. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  3. 2017Interpretation of Neural Networks is Fragilecited by 444
  4. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  5. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 302
  6. 2019BCN20000: Dermoscopic Lesions in the Wildcited by 225
  7. 2019Explanations can be manipulated and geometry is to blamecited by 216
  8. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 214
  9. 2020Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotationcited by 128
  10. 2020Reliable Post hoc Explanations: Modeling Uncertainty in Explainabilitycited by 125
  11. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120
  12. 2021HIVE: Evaluating the Human Interpretability of Visual Explanationscited by 105
  13. 2021Interpretable Image Classification with Differentiable Prototypes Assignmentcited by 97
  14. 2020On the Tractability of SHAP Explanationscited by 92
  15. 2019Lung and Colon Cancer Histopathological Image Dataset (LC25000)cited by 89
  16. 2020ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discoverycited by 89
  17. 2021This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networkscited by 65
  18. 2022ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognitioncited by 63
  19. 2019DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measurescited by 56
  20. 2022Evaluation of post-hoc interpretability methods in time-series classificationcited by 53
  21. 2023FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methodscited by 50
  22. 2021This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagationcited by 43
  23. 2023Leveraging Hidden Positives for Unsupervised Semantic Segmentationcited by 40
  24. 2023Learning Support and Trivial Prototypes for Interpretable Image Classificationcited by 38
  25. 2022Evaluation and Improvement of Interpretability for Self-Explainable Part-Prototype Networkscited by 37
  26. 2023This Looks Like Those: Illuminating Prototypical Concepts Using Multiple Visualizationscited by 36
  27. 2022ProtoVAE: A Trustworthy Self-Explainable Prototypical Variational Modelcited by 34
  28. 2023Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanationscited by 27
  29. 2023Pixel-Grounded Prototypical Part Networkscited by 18
  30. 2023Causal Unsupervised Semantic Segmentationcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 1.

8 papers.

  1. 2024Explainable Natural Language Processing for Corporate Sustainability Analysiscited by 9
  2. 2024Explain the Black Box for the Sake of Science: the Scientific Method in the Era of Generative Artificial Intelligencecited by 5
  3. 2024Revisiting the robustness of post-hoc interpretability methodscited by 5
  4. 2025XAI4Extremes: An interpretable machine learning framework for understanding extreme-weather precursors under climate changecited by 3
  5. 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
  6. 2025Interpretable Few-Shot Image Classification via Prototypical Concept-Guided Mixture of LoRA Expertscited by 1
  7. 2025This EEG Looks Like These EEGs: Interpretable Interictal Epileptiform Discharge Detection With ProtoEEG-kNNcited by 1
  8. 2026ProtoQuant: Quantization of Prototypical Parts For General and Fine-Grained Image Classificationcited by 0