# Just add noise: Debiasing tree-based variable importance in mixed data

arXiv: https://arxiv.org/abs/2609.14083 · HTML: https://mlsanity.com/paper/2609.14083/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 12 Sep 2026.

## Builds on (most cited first, 8 of 8)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [Unbiased Measurement of Feature Importance in Tree-Based Methods](https://mlsanity.com/paper/1903.05179/) (2019)
- [A Debiased MDI Feature Importance Measure for Random Forests](https://mlsanity.com/paper/1906.10845/) (2019)
- [Trees, forests, and impurity-based variable importance](https://mlsanity.com/paper/2001.04295/) (2020)
- [Integrated path stability selection](https://mlsanity.com/paper/2403.15877/) (2024)
- [Nonparametric IPSS: Fast, flexible feature selection with false discovery control](https://mlsanity.com/paper/2410.02208/) (2024)

## Abstract (arXiv)

Variable importance scores from tree-based methods such as random forests favor continuous predictors over categorical ones. We present a theoretical analysis of this bias and propose a simple remedy: add a small amount of noise to each categorical predictor. The correction is demonstrated on a variety of simulated and real-world datasets and combined with integrated path stability selection to perform variable selection with false discovery control for mixed data.