# Unbiased variable importance for random forests

arXiv: https://arxiv.org/abs/2003.02106 · HTML: https://mlsanity.com/paper/2003.02106/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 11 of 11)

- [MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA](https://mlsanity.com/paper/2102.13347/) (2021)
- [Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning](https://mlsanity.com/paper/2102.12328/) (2021)
- [Individualized and Global Feature Attributions for Gradient Boosted Trees in the Presence of ℓ2 Regularization](https://mlsanity.com/paper/2211.04409/) (2022)
- [FACT: High-Dimensional Random Forests Inference](https://mlsanity.com/paper/2207.01678/) (2022)
- [Data-driven advice for interpreting local and global model predictions in bioinformatics problems](https://mlsanity.com/paper/2108.06201/) (2021)
- [Sequential Permutation Testing of Random Forest Variable Importance Measures](https://mlsanity.com/paper/2206.01284/) (2022)
- [CIMLA: Interpretable AI for inference of differential causal networks](https://mlsanity.com/paper/2304.12523/) (2023)
- [Multi-rules mining algorithm for combinatorially exploded decision trees with modified Aitchison-Aitken function-based Bayesian optimization](https://mlsanity.com/paper/2310.02633/) (2023)
- [Multi forests: Variable importance for multi-class outcomes](https://mlsanity.com/paper/2409.08925/) (2024)
- [Provable Recovery of Locally Important Signed Features and Interactions from Random Forest](https://mlsanity.com/paper/2512.11081/) (2025)
- [Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests](https://mlsanity.com/paper/2601.07003/) (2026)

## Abstract (arXiv)

The default variable-importance measure in random Forests, Gini importance, has been shown to suffer from the bias of the underlying Gini-gain splitting criterion. While the alternative permutation importance is generally accepted as a reliable measure of variable importance, it is also computationally demanding and suffers from other shortcomings. We propose a simple solution to the misleading/untrustworthy Gini importance which can be viewed as an overfitting problem: we compute the loss reduction on the out-of-bag instead of the in-bag training samples.