# Interpreting CNNs via Decision Trees

arXiv: https://arxiv.org/abs/1802.00121 · HTML: https://mlsanity.com/paper/1802.00121/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2018.

## Built on by (most cited first, 30 of 89)

- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond](https://mlsanity.com/paper/2103.10689/) (2021)
- [Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions](https://mlsanity.com/paper/2112.11561/) (2021)
- [ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognition](https://mlsanity.com/paper/2208.10431/) (2022)
- [On Guaranteed Optimal Robust Explanations for NLP Models](https://mlsanity.com/paper/2105.03640/) (2021)
- [VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers](https://mlsanity.com/paper/2203.17247/) (2022)
- [FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods](https://mlsanity.com/paper/2308.06248/) (2023)
- [Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning](https://mlsanity.com/paper/2304.03916/) (2023)
- [Explaining Deepfake Detection by Analysing Image Matching](https://mlsanity.com/paper/2207.09679/) (2022)
- [Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI](https://mlsanity.com/paper/2302.00799/) (2023)
- [Explainable AI for Robot Failures: Generating Explanations that Improve User Assistance in Fault Recovery](https://mlsanity.com/paper/2101.01625/) (2021)
- [TopicFM: Robust and Interpretable Topic-Assisted Feature Matching](https://mlsanity.com/paper/2207.00328/) (2022)
- [Towards Interpretable Deep Metric Learning with Structural Matching](https://mlsanity.com/paper/2108.05889/) (2021)
- [Explainable Person Re-Identification with Attribute-guided Metric Distillation](https://mlsanity.com/paper/2103.01451/) (2021)
- [Causal Reasoning Meets Visual Representation Learning: A Prospective Study](https://mlsanity.com/paper/2204.12037/) (2022)
- [Visual Concept Connectome (VCC): Open World Concept Discovery and their Interlayer Connections in Deep Models](https://mlsanity.com/paper/2404.02233/) (2024)
- [Discovering Spatio-Temporal Rationales for Video Question Answering](https://mlsanity.com/paper/2307.12058/) (2023)
- [CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models](https://mlsanity.com/paper/2109.01401/) (2021)
- [Aerial-Ground Person Re-ID](https://mlsanity.com/paper/2303.08597/) (2023)
- [Interpreting Unfairness in Graph Neural Networks via Training Node Attribution](https://mlsanity.com/paper/2211.14383/) (2022)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks](https://mlsanity.com/paper/2103.07224/) (2021)
- [HINT: Hierarchical Neuron Concept Explainer](https://mlsanity.com/paper/2203.14196/) (2022)
- [Robust Semantic Interpretability: Revisiting Concept Activation Vectors](https://mlsanity.com/paper/2104.02768/) (2021)
- [Collaborative Semantic Aggregation and Calibration for Federated Domain Generalization](https://mlsanity.com/paper/2110.06736/) (2021)
- [A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks](https://mlsanity.com/paper/2102.01792/) (2021)
- [Explainable Activity Recognition for Smart Home Systems](https://mlsanity.com/paper/2105.09787/) (2021)
- [Interpretability of Machine Learning: Recent Advances and Future Prospects](https://mlsanity.com/paper/2305.00537/) (2023)

## Abstract (arXiv)

This paper aims to quantitatively explain rationales of each prediction that is made by a pre-trained convolutional neural network (CNN). We propose to learn a decision tree, which clarifies the specific reason for each prediction made by the CNN at the semantic level. I.e., the decision tree decomposes feature representations in high conv-layers of the CNN into elementary concepts of object parts. In this way, the decision tree tells people which object parts activate which filters for the prediction and how much they contribute to the prediction score. Such semantic and quantitative explanations for CNN predictions have specific values beyond the traditional pixel-level analysis of CNNs. More specifically, our method mines all potential decision modes of the CNN, where each mode represents a common case of how the CNN uses object parts for prediction. The decision tree organizes all potential decision modes in a coarse-to-fine manner to explain CNN predictions at different fine-grained levels. Experiments have demonstrated the effectiveness of the proposed method.