# Interpreting Deep Forest through Feature Contribution and MDI Feature Importance

arXiv: https://arxiv.org/abs/2305.00805 · HTML: https://mlsanity.com/paper/2305.00805/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 46 references that are in the map.
- First posted on arXiv 1 May 2023.

## Builds on (most cited first, 17 of 17)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Why do tree-based models still outperform deep learning on tabular data?](https://mlsanity.com/paper/2207.08815/) (2022)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [Deep Forest](https://mlsanity.com/paper/1702.08835/) (2017)
- [Iterative Random Forests to detect predictive and stable high-order interactions](https://mlsanity.com/paper/1706.08457/) (2017)
- [Unbiased Measurement of Feature Importance in Tree-Based Methods](https://mlsanity.com/paper/1903.05179/) (2019)
- [A Debiased MDI Feature Importance Measure for Random Forests](https://mlsanity.com/paper/1906.10845/) (2019)
- [Multi-Label Learning with Deep Forest](https://mlsanity.com/paper/1911.06557/) (2019)
- [Distributed Deep Forest and its Application to Automatic Detection of Cash-out Fraud](https://mlsanity.com/paper/1805.04234/) (2018)
- [Interpreting random forest classification models using a feature contribution method](https://mlsanity.com/paper/1312.1121/) (2013)
- [Analyzing the tree-layer structure of Deep Forests](https://mlsanity.com/paper/2010.15690/) (2020)
- [A Siamese Deep Forest](https://mlsanity.com/paper/1704.08715/) (2017)
- [Discriminative Metric Learning with Deep Forest](https://mlsanity.com/paper/1705.09620/) (2017)
- [Deep Forest with Hashing Screening and Window Screening](https://mlsanity.com/paper/2207.11951/) (2022)
- [Context-dependent feature analysis with random forests](https://mlsanity.com/paper/1605.03848/) (2016)
- [Interpretation and Simplification of Deep Forest](https://mlsanity.com/paper/2001.04721/) (2020)

## Built on by (most cited first, 1 of 1)

- [Enhancing Job Salary Prediction with Disentangled Composition Effect Modeling: A Neural Prototyping Approach](https://mlsanity.com/paper/2503.12978/) (2025)

## Abstract (arXiv)

Deep forest is a non-differentiable deep model which has achieved impressive empirical success across a wide variety of applications, especially on categorical/symbolic or mixed modeling tasks. Many of the application fields prefer explainable models, such as random forests with feature contributions that can provide local explanation for each prediction, and Mean Decrease Impurity (MDI) that can provide global feature importance. However, deep forest, as a cascade of random forests, possesses interpretability only at the first layer. From the second layer on, many of the tree splits occur on the new features generated by the previous layer, which makes existing explanatory tools for random forests inapplicable. To disclose the impact of the original features in the deep layers, we design a calculation method with an estimation step followed by a calibration step for each layer, and propose our feature contribution and MDI feature importance calculation tools for deep forest. Experimental results on both simulated data and real world data verify the effectiveness of our methods.