# WildWood: a new Random Forest algorithm

arXiv: https://arxiv.org/abs/2109.08010 · HTML: https://mlsanity.com/paper/2109.08010/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 100 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 26 of 26)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [CatBoost: gradient boosting with categorical features support](https://mlsanity.com/paper/1810.11363/) (2018)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [Hyperparameters and Tuning Strategies for Random Forest](https://mlsanity.com/paper/1804.03515/) (2018)
- [Understanding Random Forests: From Theory to Practice](https://mlsanity.com/paper/1407.7502/) (2014)
- [Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success](https://mlsanity.com/paper/1911.00190/) (2019)
- [Mondrian Forests: Efficient Online Random Forests](https://mlsanity.com/paper/1406.2673/) (2014)
- [Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features](https://mlsanity.com/paper/2104.00629/) (2021)
- [Minimax optimal rates for Mondrian trees and forests](https://mlsanity.com/paper/1803.05784/) (2018)
- [AMF: Aggregated Mondrian Forests for Online Learning](https://mlsanity.com/paper/1906.10529/) (2019)
- [Trees, Forests, Chickens, and Eggs: When and Why to Prune Trees in a Random Forest](https://mlsanity.com/paper/2103.16700/) (2021)
- [Analysis of purely random forests bias](https://mlsanity.com/paper/1407.3939/) (2014)
- [Approximation Trees: Statistical Stability in Model Distillation](https://mlsanity.com/paper/1808.07573/) (2018)
- [Predicting Intensive Care Unit Length of Stay and Mortality Using Patient Vital Signs: Machine Learning Model Development and Validation](https://mlsanity.com/paper/2105.04414/) (2021)
- [Random Forest for Label Ranking](https://mlsanity.com/paper/1608.07710/) (2016)
- [Forest Guided Smoothing](https://mlsanity.com/paper/2103.05092/) (2021)
- [Dimension Reduction Forests: Local Variable Importance using Structured Random Forests](https://mlsanity.com/paper/2103.13233/) (2021)
- [ADASYN-Random Forest Based Intrusion Detection Model](https://mlsanity.com/paper/2105.04301/) (2021)
- [Generalised Boosted Forests](https://mlsanity.com/paper/2102.12561/) (2021)
- [Driver State and Behavior Detection Through Smart Wearables](https://mlsanity.com/paper/2104.13889/) (2021)
- [Estimating County-Level COVID-19 Exponential Growth Rates Using Generalized Random Forests](https://mlsanity.com/paper/2011.01219/) (2020)
- [Multi-Output Random Forest Regression to Emulate the Earliest Stages of Planet Formation](https://mlsanity.com/paper/2104.12845/) (2021)

## Abstract (arXiv)

We introduce WildWood (WW), a new ensemble algorithm for supervised learning of Random Forest (RF) type. While standard RF algorithms use bootstrap out-of-bag samples to compute out-of-bag scores, WW uses these samples to produce improved predictions given by an aggregation of the predictions of all possible subtrees of each fully grown tree in the forest. This is achieved by aggregation with exponential weights computed over out-of-bag samples, that are computed exactly and very efficiently thanks to an algorithm called context tree weighting. This improvement, combined with a histogram strategy to accelerate split finding, makes WW fast and competitive compared with other well-established ensemble methods, such as standard RF and extreme gradient boosting algorithms.