# Making CNNs Interpretable by Building Dynamic Sequential Decision Forests with Top-down Hierarchy Learning

arXiv: https://arxiv.org/abs/2106.02824 · HTML: https://mlsanity.com/paper/2106.02824/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 5 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Distilling a Neural Network Into a Soft Decision Tree](https://mlsanity.com/paper/1711.09784/) (2017)
- [Few-Shot Image Recognition by Predicting Parameters from Activations](https://mlsanity.com/paper/1706.03466/) (2017)
- [Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs](https://mlsanity.com/paper/2003.00152/) (2020)
- [Interpreting CNNs via Decision Trees](https://mlsanity.com/paper/1802.00121/) (2018)
- [Adaptive Neural Trees](https://mlsanity.com/paper/1807.06699/) (2018)
- [Born-Again Tree Ensembles](https://mlsanity.com/paper/2003.11132/) (2020)
- [Deep Regression Forests for Age Estimation](https://mlsanity.com/paper/1712.07195/) (2017)
- [Tree-CNN: A Hierarchical Deep Convolutional Neural Network for Incremental Learning](https://mlsanity.com/paper/1802.05800/) (2018)
- [Label Distribution Learning Forests](https://mlsanity.com/paper/1702.06086/) (2017)
- [Deep Nets: What have they ever done for Vision?](https://mlsanity.com/paper/1805.04025/) (2018)
- [Interpretable Random Forests via Rule Extraction](https://mlsanity.com/paper/2004.14841/) (2020)
- [Deep Differentiable Random Forests for Age Estimation](https://mlsanity.com/paper/1907.10665/) (2019)
- [Decision Forests, Convolutional Networks and the Models in-Between](https://mlsanity.com/paper/1603.01250/) (2016)
- [Self-Paced Deep Regression Forests with Consideration on Underrepresented Examples](https://mlsanity.com/paper/2004.01459/) (2020)
- [Deep Embedding Forest: Forest-based Serving with Deep Embedding Features](https://mlsanity.com/paper/1703.05291/) (2017)
- [Learning Decision Trees Recurrently Through Communication](https://mlsanity.com/paper/1902.01780/) (2019)

## Abstract (arXiv)

In this paper, we propose a generic model transfer scheme to make Convlutional Neural Networks (CNNs) interpretable, while maintaining their high classification accuracy. We achieve this by building a differentiable decision forest on top of CNNs, which enjoys two characteristics: 1) During training, the tree hierarchies of the forest are learned in a top-down manner under the guidance from the category semantics embedded in the pre-trained CNN weights; 2) During inference, a single decision tree is dynamically selected from the forest for each input sample, enabling the transferred model to make sequential decisions corresponding to the attributes shared by semantically-similar categories, rather than directly performing flat classification. We name the transferred model deep Dynamic Sequential Decision Forest (dDSDF). Experimental results show that dDSDF not only achieves higher classification accuracy than its conuterpart, i.e., the original CNN, but has much better interpretability, as qualitatively it has plausible hierarchies and quantitatively it leads to more precise saliency maps.