# Semantic Prototypes: Enhancing Transparency Without Black Boxes

arXiv: https://arxiv.org/abs/2407.15871 · HTML: https://mlsanity.com/paper/2407.15871/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Neural Prototype Trees for Interpretable Fine-grained Image Recognition](https://mlsanity.com/paper/2012.02046/) (2020)
- [Explaining Explanations in AI](https://mlsanity.com/paper/1811.01439/) (2018)
- [Interpretable Image Classification with Differentiable Prototypes Assignment](https://mlsanity.com/paper/2112.02902/) (2021)
- [Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations](https://mlsanity.com/paper/2011.12854/) (2020)
- [ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery](https://mlsanity.com/paper/2011.14340/) (2020)
- [This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networks](https://mlsanity.com/paper/2105.02968/) (2021)
- [ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognition](https://mlsanity.com/paper/2208.10431/) (2022)
- ["Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction](https://mlsanity.com/paper/2210.03735/) (2022)
- [This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition](https://mlsanity.com/paper/2011.02863/) (2020)
- [Choose your Data Wisely: A Framework for Semantic Counterfactuals](https://mlsanity.com/paper/2305.17667/) (2023)
- [Structure Your Data: Towards Semantic Graph Counterfactuals](https://mlsanity.com/paper/2403.06514/) (2024)
- [Towards Deep Machine Reasoning: a Prototype-based Deep Neural Network with Decision Tree Inference](https://mlsanity.com/paper/2002.03776/) (2020)

## Built on by (most cited first, 3 of 3)

- [MusicLIME: Explainable Multimodal Music Understanding](https://mlsanity.com/paper/2409.10496/) (2024)
- [This part looks alike this: identifying important parts of explained instances and prototypes](https://mlsanity.com/paper/2505.05597/) (2025)
- [Alike Parts: A Feature-Informed Approach to Local and Global Prototype Explanations](https://mlsanity.com/paper/2605.21646/) (2026)

## Abstract (arXiv)

As machine learning (ML) models and datasets increase in complexity, the demand for methods that enhance explainability and interpretability becomes paramount. Prototypes, by encapsulating essential characteristics within data, offer insights that enable tactical decision-making and enhance transparency. Traditional prototype methods often rely on sub-symbolic raw data and opaque latent spaces, reducing explainability and increasing the risk of misinterpretations. This paper presents a novel framework that utilizes semantic descriptions to define prototypes and provide clear explanations, effectively addressing the shortcomings of conventional methods. Our approach leverages concept-based descriptions to cluster data on the semantic level, ensuring that prototypes not only represent underlying properties intuitively but are also straightforward to interpret. Our method simplifies the interpretative process and effectively bridges the gap between complex data structures and human cognitive processes, thereby enhancing transparency and fostering trust. Our approach outperforms existing widely-used prototype methods in facilitating human understanding and informativeness, as validated through a user survey.