# Unbiased Measurement of Feature Importance in Tree-Based Methods

arXiv: https://arxiv.org/abs/1903.05179 · HTML: https://mlsanity.com/paper/1903.05179/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2019.

## Built on by (most cited first, 30 of 30)

- [S-LIME: Stabilized-LIME for Model Explanation](https://mlsanity.com/paper/2106.07875/) (2021)
- [Fast Interpretable Greedy-Tree Sums](https://mlsanity.com/paper/2201.11931/) (2022)
- [MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA](https://mlsanity.com/paper/2102.13347/) (2021)
- [ControlBurn: Feature Selection by Sparse Forests](https://mlsanity.com/paper/2107.00219/) (2021)
- [Feature Importance Explanations for Temporal Black-Box Models](https://mlsanity.com/paper/2102.11934/) (2021)
- [Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning](https://mlsanity.com/paper/2102.12328/) (2021)
- [GlyTwin: Digital Twin for Glucose Control in Type 1 Diabetes Through Optimal Behavioral Modifications Using Patient-Centric Counterfactuals](https://mlsanity.com/paper/2504.09846/) (2025)
- [Individualized and Global Feature Attributions for Gradient Boosted Trees in the Presence of ℓ2 Regularization](https://mlsanity.com/paper/2211.04409/) (2022)
- [Unbiased Gradient Boosting Decision Tree with Unbiased Feature Importance](https://mlsanity.com/paper/2305.10696/) (2023)
- [Modeling Household Online Shopping Demand in the U.S.: A Machine Learning Approach and Comparative Investigation between 2009 and 2017](https://mlsanity.com/paper/2101.03690/) (2021)
- [A Subspace-based Approach for Dimensionality Reduction and Important Variable Selection](https://mlsanity.com/paper/2106.01584/) (2021)
- [ControlBurn: Nonlinear Feature Selection with Sparse Tree Ensembles](https://mlsanity.com/paper/2207.03935/) (2022)
- [Interpreting Deep Forest through Feature Contribution and MDI Feature Importance](https://mlsanity.com/paper/2305.00805/) (2023)
- [McUDI: Model-Centric Unsupervised Degradation Indicator for Failure Prediction AIOps Solutions](https://mlsanity.com/paper/2401.14093/) (2024)
- [Tackling Small Sample Survival Analysis via Transfer Learning: A Study of Colorectal Cancer Prognosis](https://mlsanity.com/paper/2501.12421/) (2025)
- [Machine learning for detection of stenoses and aneurysms: application in a physiologically realistic virtual patient database](https://mlsanity.com/paper/2103.00599/) (2021)
- [A Novel Splitting Criterion Inspired by Geometric Mean Metric Learning for Decision Tree](https://mlsanity.com/paper/2204.11011/) (2022)
- [The Berkelmans-Pries Feature Importance Method: A Generic Measure of Informativeness of Features](https://mlsanity.com/paper/2301.04740/) (2023)
- [CIMLA: Interpretable AI for inference of differential causal networks](https://mlsanity.com/paper/2304.12523/) (2023)
- [End-to-end Feature Selection Approach for Learning Skinny Trees](https://mlsanity.com/paper/2310.18542/) (2023)
- [Evaluating Explanatory Capabilities of Machine Learning Models in Medical Diagnostics: A Human-in-the-Loop Approach](https://mlsanity.com/paper/2403.19820/) (2024)
- [Predicting the duration of traffic incidents for Sydney greater metropolitan area using machine learning methods](https://mlsanity.com/paper/2406.18861/) (2024)
- [Multi forests: Variable importance for multi-class outcomes](https://mlsanity.com/paper/2409.08925/) (2024)
- [Iterative Feature Exclusion Ranking for Deep Tabular Learning](https://mlsanity.com/paper/2412.16442/) (2024)
- [Provable Recovery of Locally Important Signed Features and Interactions from Random Forest](https://mlsanity.com/paper/2512.11081/) (2025)
- [Towards Public Administration Research Based on Interpretable Machine Learning](https://mlsanity.com/paper/2601.06205/) (2026)
- [Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests](https://mlsanity.com/paper/2601.07003/) (2026)
- [SeqShield: A Behavioral Analysis Approach to Uncover Rootkits](https://mlsanity.com/paper/2604.23812/) (2026)
- [Just add noise: Debiasing tree-based variable importance in mixed data](https://mlsanity.com/paper/2609.14083/) (2026)
- [Splitting the Difference: Interpretable Causal Forests for Treatment Effect Heterogeneity and Bias](https://mlsanity.com/paper/2609.16971/) (2026)

## Abstract (arXiv)

We propose a modification that corrects for split-improvement variable importance measures in Random Forests and other tree-based methods. These methods have been shown to be biased towards increasing the importance of features with more potential splits. We show that by appropriately incorporating split-improvement as measured on out of sample data, this bias can be corrected yielding better summaries and screening tools.