# A Debiased MDI Feature Importance Measure for Random Forests

arXiv: https://arxiv.org/abs/1906.10845 · HTML: https://mlsanity.com/paper/1906.10845/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2019.

## Built on by (most cited first, 24 of 24)

- [MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA](https://mlsanity.com/paper/2102.13347/) (2021)
- [Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning](https://mlsanity.com/paper/2102.12328/) (2021)
- [From global to local MDI variable importances for random forests and when they are Shapley values](https://mlsanity.com/paper/2111.02218/) (2021)
- [Bias, Fairness, and Accountability with AI and ML Algorithms](https://mlsanity.com/paper/2105.06558/) (2021)
- [Nonparametric Feature Selection by Random Forests and Deep Neural Networks](https://mlsanity.com/paper/2201.06821/) (2022)
- [Individualized and Global Feature Attributions for Gradient Boosted Trees in the Presence of ℓ2 Regularization](https://mlsanity.com/paper/2211.04409/) (2022)
- [Optimal Sparse Recovery with Decision Stumps](https://mlsanity.com/paper/2303.04301/) (2023)
- [Unbiased Gradient Boosting Decision Tree with Unbiased Feature Importance](https://mlsanity.com/paper/2305.10696/) (2023)
- [FACT: High-Dimensional Random Forests Inference](https://mlsanity.com/paper/2207.01678/) (2022)
- [Interpreting Deep Forest through Feature Contribution and MDI Feature Importance](https://mlsanity.com/paper/2305.00805/) (2023)
- [McUDI: Model-Centric Unsupervised Degradation Indicator for Failure Prediction AIOps Solutions](https://mlsanity.com/paper/2401.14093/) (2024)
- [Online High-Frequency Trading Stock Forecasting with Automated Feature Clustering and Radial Basis Function Neural Networks](https://mlsanity.com/paper/2412.16160/) (2024)
- [Data-driven advice for interpreting local and global model predictions in bioinformatics problems](https://mlsanity.com/paper/2108.06201/) (2021)
- [How important are socioeconomic factors for hurricane performance of power systems? An analysis of disparities through machine learning](https://mlsanity.com/paper/2208.09063/) (2022)
- [The Berkelmans-Pries Feature Importance Method: A Generic Measure of Informativeness of Features](https://mlsanity.com/paper/2301.04740/) (2023)
- [CIMLA: Interpretable AI for inference of differential causal networks](https://mlsanity.com/paper/2304.12523/) (2023)
- [Analyze Additive and Interaction Effects via Collaborative Trees](https://mlsanity.com/paper/2405.11477/) (2024)
- [Regression Trees Know Calculus](https://mlsanity.com/paper/2405.13846/) (2024)
- [Multi forests: Variable importance for multi-class outcomes](https://mlsanity.com/paper/2409.08925/) (2024)
- [Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models](https://mlsanity.com/paper/2502.07153/) (2025)
- [Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests](https://mlsanity.com/paper/2601.07003/) (2026)
- [Alike Parts: A Feature-Informed Approach to Local and Global Prototype Explanations](https://mlsanity.com/paper/2605.21646/) (2026)
- [Just add noise: Debiasing tree-based variable importance in mixed data](https://mlsanity.com/paper/2609.14083/) (2026)
- [Splitting the Difference: Interpretable Causal Forests for Treatment Effect Heterogeneity and Bias](https://mlsanity.com/paper/2609.16971/) (2026)

## Abstract (arXiv)

Tree ensembles such as Random Forests have achieved impressive empirical success across a wide variety of applications. To understand how these models make predictions, people routinely turn to feature importance measures calculated from tree ensembles. It has long been known that Mean Decrease Impurity (MDI), one of the most widely used measures of feature importance, incorrectly assigns high importance to noisy features, leading to systematic bias in feature selection. In this paper, we address the feature selection bias of MDI from both theoretical and methodological perspectives. Based on the original definition of MDI by Breiman et al. for a single tree, we derive a tight non-asymptotic bound on the expected bias of MDI importance of noisy features, showing that deep trees have higher (expected) feature selection bias than shallow ones. However, it is not clear how to reduce the bias of MDI using its existing analytical expression. We derive a new analytical expression for MDI, and based on this new expression, we are able to propose a debiased MDI feature importance measure using out-of-bag samples, called MDI-oob. For both the simulated data and a genomic ChIP dataset, MDI-oob achieves state-of-the-art performance in feature selection from Random Forests for both deep and shallow trees.