# Explaining Classifiers by Constructing Familiar Concepts

arXiv: https://arxiv.org/abs/2203.04109 · HTML: https://mlsanity.com/paper/2203.04109/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 28 of 28)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [On the (In)fidelity and Sensitivity for Explanations](https://mlsanity.com/paper/1901.09392/) (2019)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Synthesizing the preferred inputs for neurons in neural networks via deep generator networks](https://mlsanity.com/paper/1605.09304/) (2016)
- [Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks](https://mlsanity.com/paper/1801.03454/) (2018)
- [Towards Visually Explaining Variational Autoencoders](https://mlsanity.com/paper/1911.07389/) (2019)
- [On Information Plane Analyses of Neural Network Classifiers -- A Review](https://mlsanity.com/paper/2003.09671/) (2020)
- [Evaluating Explanation Without Ground Truth in Interpretable Machine Learning](https://mlsanity.com/paper/1907.06831/) (2019)
- [Black Box Explanation by Learning Image Exemplars in the Latent Feature Space](https://mlsanity.com/paper/2002.03746/) (2020)
- [Deceptive AI Explanations: Creation and Detection](https://mlsanity.com/paper/2001.07641/) (2020)
- [Reflective-Net: Learning from Explanations](https://mlsanity.com/paper/2011.13986/) (2020)
- [Understanding trained CNNs by indexing neuron selectivity](https://mlsanity.com/paper/1702.00382/) (2017)
- [Trepan Reloaded: A Knowledge-driven Approach to Explaining Artificial Neural Networks](https://mlsanity.com/paper/1906.08362/) (2019)
- [A Generic and Model-Agnostic Exemplar Synthetization Framework for Explainable AI](https://mlsanity.com/paper/2006.03896/) (2020)
- [Human-to-AI Coach: Improving Human Inputs to AI Systems](https://mlsanity.com/paper/1912.03652/) (2019)
- [On Randomly Projected Hierarchical Clustering with Guarantees](https://mlsanity.com/paper/1401.5814/) (2014)

## Built on by (most cited first, 7 of 7)

- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda](https://mlsanity.com/paper/2404.09554/) (2024)
- [Concept-based Adversarial Attacks: Tricking Humans and Classifiers Alike](https://mlsanity.com/paper/2203.10166/) (2022)
- [Cultural Bias in Explainable AI Research: A Systematic Analysis](https://mlsanity.com/paper/2403.05579/) (2024)
- [Understanding and Leveraging the Learning Phases of Neural Networks](https://mlsanity.com/paper/2312.06887/) (2023)
- [SIDEs: Separating Idealization from Deceptive Explanations in xAI](https://mlsanity.com/paper/2404.16534/) (2024)
- [Improving classifier decision boundaries using nearest neighbors](https://mlsanity.com/paper/2310.03927/) (2023)

## Abstract (arXiv)

Interpreting a large number of neurons in deep learning is difficult. Our proposed `CLAssifier-DECoder' architecture (ClaDec) facilitates the understanding of the output of an arbitrary layer of neurons or subsets thereof. It uses a decoder that transforms the incomprehensible representation of the given neurons to a representation that is more similar to the domain a human is familiar with. In an image recognition problem, one can recognize what information (or concepts) a layer maintains by contrasting reconstructed images of ClaDec with those of a conventional auto-encoder(AE) serving as reference. An extension of ClaDec allows trading comprehensibility and fidelity. We evaluate our approach for image classification using convolutional neural networks. We show that reconstructed visualizations using encodings from a classifier capture more relevant classification information than conventional AEs. This holds although AEs contain more information on the original input. Our user study highlights that even non-experts can identify a diverse set of concepts contained in images that are relevant (or irrelevant) for the classifier. We also compare against saliency based methods that focus on pixel relevance rather than concepts. We show that ClaDec tends to highlight more relevant input areas to classification though outcomes depend on classifier architecture. Code is at urlhttps://github.com/JohnTailor/ClaDec