# "I know it when I see it". Visualization and Intuitive Interpretability

arXiv: https://arxiv.org/abs/1711.08042 · HTML: https://mlsanity.com/paper/1711.08042/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2017.

## Built on by (most cited first, 4 of 4)

- [Keep Your Friends Close and Your Counterfactuals Closer: Improved Learning From Closest Rather Than Plausible Counterfactual Explanations in an Abstract Setting](https://mlsanity.com/paper/2205.05515/) (2022)
- ["Even if ..." -- Diverse Semifactual Explanations of Reject](https://mlsanity.com/paper/2207.01898/) (2022)
- [Let's Go to the Alien Zoo: Introducing an Experimental Framework to Study Usability of Counterfactual Explanations for Machine Learning](https://mlsanity.com/paper/2205.03398/) (2022)
- [One Explanation to Rule them All -- Ensemble Consistent Explanations](https://mlsanity.com/paper/2205.08974/) (2022)

## Abstract (arXiv)

Most research on the interpretability of machine learning systems focuses on the development of a more rigorous notion of interpretability. I suggest that a better understanding of the deficiencies of the intuitive notion of interpretability is needed as well. I show that visualization enables but also impedes intuitive interpretability, as it presupposes two levels of technical pre-interpretation: dimensionality reduction and regularization. Furthermore, I argue that the use of positive concepts to emulate the distributed semantic structure of machine learning models introduces a significant human bias into the model. As a consequence, I suggest that, if intuitive interpretability is needed, singular representations of internal model states should be avoided.