# Interpreting Deep Classifier by Visual Distillation of Dark Knowledge

arXiv: https://arxiv.org/abs/1803.04042 · HTML: https://mlsanity.com/paper/1803.04042/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Mar 2018.

## Built on by (most cited first, 9 of 9)

- [Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey](https://mlsanity.com/paper/2104.00950/) (2021)
- [Proto2Proto: Can you recognize the car, the way I do?](https://mlsanity.com/paper/2204.11830/) (2022)
- [Random Teachers are Good Teachers](https://mlsanity.com/paper/2302.12091/) (2023)
- [Adam: Dense Retrieval Distillation with Adaptive Dark Examples](https://mlsanity.com/paper/2212.10192/) (2022)
- [Understanding with toy surrogate models in machine learning](https://mlsanity.com/paper/2410.05675/) (2024)
- [Explainable AI the Latest Advancements and New Trends](https://mlsanity.com/paper/2505.07005/) (2025)
- [A Machine Learning Framework for Distributed Functional Compression over Wireless Channels in IoT](https://mlsanity.com/paper/2201.09483/) (2022)
- [Deep Explainable Learning with Graph Based Data Assessing and Rule Reasoning](https://mlsanity.com/paper/2211.04693/) (2022)
- [Semiotics Networks Representing Perceptual Inference](https://mlsanity.com/paper/2310.05212/) (2023)

## Abstract (arXiv)

Interpreting black box classifiers, such as deep networks, allows an analyst to validate a classifier before it is deployed in a high-stakes setting. A natural idea is to visualize the deep network's representations, so as to "see what the network sees". In this paper, we demonstrate that standard dimension reduction methods in this setting can yield uninformative or even misleading visualizations. Instead, we present DarkSight, which visually summarizes the predictions of a classifier in a way inspired by notion of dark knowledge. DarkSight embeds the data points into a low-dimensional space such that it is easy to compress the deep classifier into a simpler one, essentially combining model compression and dimension reduction. We compare DarkSight against t-SNE both qualitatively and quantitatively, demonstrating that DarkSight visualizations are more informative. Our method additionally yields a new confidence measure based on dark knowledge by quantifying how unusual a given vector of predictions is.