# Random Forests, Decision Trees, and Categorical Predictors: The "Absent Levels" Problem

arXiv: https://arxiv.org/abs/1706.03492 · HTML: https://mlsanity.com/paper/1706.03492/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2017.

## Built on by (most cited first, 5 of 5)

- [Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees](https://mlsanity.com/paper/2204.07293/) (2022)
- [Predicting Pollution Level Using Random Forest: A Case Study of Marilao River in Bulacan Province, Philippines](https://mlsanity.com/paper/2202.06066/) (2022)
- [Design Principles of Robust Multi-Armed Bandit Framework in Video Recommendations](https://mlsanity.com/paper/2310.01419/) (2023)
- [Machine learning-based algorithms for at-home respiratory disease monitoring and respiratory assessment](https://mlsanity.com/paper/2409.03180/) (2024)
- [Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data](https://mlsanity.com/paper/2604.05857/) (2026)

## Abstract (arXiv)

One advantage of decision tree based methods like random forests is their ability to natively handle categorical predictors without having to first transform them (e.g., by using feature engineering techniques). However, in this paper, we show how this capability can lead to an inherent "absent levels" problem for decision tree based methods that has never been thoroughly discussed, and whose consequences have never been carefully explored. This problem occurs whenever there is an indeterminacy over how to handle an observation that has reached a categorical split which was determined when the observation in question's level was absent during training. Although these incidents may appear to be innocuous, by using Leo Breiman and Adele Cutler's random forests FORTRAN code and the randomForest R package (Liaw and Wiener, 2002) as motivating case studies, we examine how overlooking the absent levels problem can systematically bias a model. Furthermore, by using three real data examples, we illustrate how absent levels can dramatically alter a model's performance in practice, and we empirically demonstrate how some simple heuristics can be used to help mitigate the effects of the absent levels problem until a more robust theoretical solution is found.