# Analysis of purely random forests bias

arXiv: https://arxiv.org/abs/1407.3939 · HTML: https://mlsanity.com/paper/1407.3939/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2014.

## Built on by (most cited first, 10 of 10)

- [Random Forest Weighted Local Fr'echet Regression with Random Objects](https://mlsanity.com/paper/2202.04912/) (2022)
- [Scaffolding Sets](https://mlsanity.com/paper/2111.03135/) (2021)
- [On the Convergence of CART under Sufficient Impurity Decrease Condition](https://mlsanity.com/paper/2310.17114/) (2023)
- [Do we need rebalancing strategies? A theoretical and empirical study around SMOTE and its variants](https://mlsanity.com/paper/2402.03819/) (2024)
- [Minimax Rates for High-Dimensional Random Tessellation Forests](https://mlsanity.com/paper/2109.10541/) (2021)
- [Statistical Advantages of Oblique Randomized Decision Trees and Forests](https://mlsanity.com/paper/2407.02458/) (2024)
- [Minimax-optimal and Locally-adaptive Online Nonparametric Regression](https://mlsanity.com/paper/2410.03363/) (2024)
- [How Many Trees in a Random Forest? A Revisited Approach with Plateau Search and Optuna Integration](https://mlsanity.com/paper/2606.03549/) (2026)
- [WildWood: a new Random Forest algorithm](https://mlsanity.com/paper/2109.08010/) (2021)
- [Consistency of Honest Decision Trees and Random Forests](https://mlsanity.com/paper/2601.14991/) (2026)

## Abstract (arXiv)

Random forests are a very effective and commonly used statistical method, but their full theoretical analysis is still an open problem. As a first step, simplified models such as purely random forests have been introduced, in order to shed light on the good performance of random forests. In this paper, we study the approximation error (the bias) of some purely random forest models in a regression framework, focusing in particular on the influence of the number of trees in the forest. Under some regularity assumptions on the regression function, we show that the bias of an infinite forest decreases at a faster rate (with respect to the size of each tree) than a single tree. As a consequence, infinite forests attain a strictly better risk rate (with respect to the sample size) than single trees. Furthermore, our results allow to derive a minimum number of trees sufficient to reach the same rate as an infinite forest. As a by-product of our analysis, we also show a link between the bias of purely random forests and the bias of some kernel estimators.