# Consistency of random forests

arXiv: https://arxiv.org/abs/1405.2881 · HTML: https://mlsanity.com/paper/1405.2881/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2014.

## Built on by (most cited first, 30 of 85)

- [Adversarial random forests for density estimation and generative modeling](https://mlsanity.com/paper/2205.09435/) (2022)
- [Large Scale Prediction with Decision Trees](https://mlsanity.com/paper/2104.13881/) (2021)
- [SHAFF: Fast and consistent SHApley eFfect estimates via random Forests](https://mlsanity.com/paper/2105.11724/) (2021)
- [Adaptive Conformal Prediction by Reweighting Nonconformity Score](https://mlsanity.com/paper/2303.12695/) (2023)
- [Trees, Forests, Chickens, and Eggs: When and Why to Prune Trees in a Random Forest](https://mlsanity.com/paper/2103.16700/) (2021)
- [Random Forest Weighted Local Fr'echet Regression with Random Objects](https://mlsanity.com/paper/2202.04912/) (2022)
- [MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA](https://mlsanity.com/paper/2102.13347/) (2021)
- [Accurate Shapley Values for explaining tree-based models](https://mlsanity.com/paper/2106.03820/) (2021)
- [A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds](https://mlsanity.com/paper/2110.09626/) (2021)
- [Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning](https://mlsanity.com/paper/2102.12328/) (2021)
- [MCCE: Monte Carlo sampling of realistic counterfactual explanations](https://mlsanity.com/paper/2111.09790/) (2021)
- [Fast Linear Model Trees by PILOT](https://mlsanity.com/paper/2302.03931/) (2023)
- [Majority of the Bests: Improving Best-of-N via Bootstrapping](https://mlsanity.com/paper/2511.18630/) (2025)
- [Variable importance for causal forests: breaking down the heterogeneity of treatment effects](https://mlsanity.com/paper/2308.03369/) (2023)
- [Inference with Mondrian Random Forests](https://mlsanity.com/paper/2310.09702/) (2023)
- [Are Ensembles Getting Better all the Time?](https://mlsanity.com/paper/2311.17885/) (2023)
- [CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk](https://mlsanity.com/paper/2507.08150/) (2025)
- [Tree Ensemble Explainability through the Hoeffding Functional Decomposition and TreeHFD Algorithm](https://mlsanity.com/paper/2510.24815/) (2025)
- [On the Convergence of CART under Sufficient Impurity Decrease Condition](https://mlsanity.com/paper/2310.17114/) (2023)
- [A universally consistent learning rule with a universally monotone error](https://mlsanity.com/paper/2108.09733/) (2021)
- [Minimax Rates for High-Dimensional Random Tessellation Forests](https://mlsanity.com/paper/2109.10541/) (2021)
- [Consistent Sufficient Explanations and Minimal Local Rules for explaining regression and classification models](https://mlsanity.com/paper/2111.04658/) (2021)
- [Meta-Learners for Estimation of Causal Effects: Finite Sample Cross-Fit Performance](https://mlsanity.com/paper/2201.12692/) (2022)
- [Extrapolated cross-validation for randomized ensembles](https://mlsanity.com/paper/2302.13511/) (2023)
- [Distributionally Robust Learning for Multi-source Unsupervised Domain Adaptation](https://mlsanity.com/paper/2309.02211/) (2023)
- [Stability of Random Forests and Coverage of Random-Forest Prediction Intervals](https://mlsanity.com/paper/2310.18814/) (2023)
- [Label Ranking through Nonparametric Regression](https://mlsanity.com/paper/2111.02749/) (2021)
- [A model-free feature selection technique of feature screening and random forest based recursive feature elimination](https://mlsanity.com/paper/2302.07449/) (2023)
- [Adaptive auditing of AI systems with anytime-valid guarantees](https://mlsanity.com/paper/2605.07002/) (2026)
- [Nonparametric Feature Selection by Random Forests and Deep Neural Networks](https://mlsanity.com/paper/2201.06821/) (2022)

## Abstract (arXiv)

Random forests are a learning algorithm proposed by Breiman [Mach. Learn. 45 (2001) 5--32] that combines several randomized decision trees and aggregates their predictions by averaging. Despite its wide usage and outstanding practical performance, little is known about the mathematical properties of the procedure. This disparity between theory and practice originates in the difficulty to simultaneously analyze both the randomization process and the highly data-dependent tree structure. In the present paper, we take a step forward in forest exploration by proving a consistency result for Breiman's [Mach. Learn. 45 (2001) 5--32] original algorithm in the context of additive regression models. Our analysis also sheds an interesting light on how random forests can nicely adapt to sparsity. 1. Introduction. Random forests are an ensemble learning method for classification and regression that constructs a number of randomized decision trees during the training phase and predicts by averaging the results. Since its publication in the seminal paper of Breiman (2001), the procedure has become a major data analysis tool, that performs well in practice in comparison with many standard methods. What has greatly contributed to the popularity of forests is the fact that they can be applied to a wide range of prediction problems and have few parameters to tune. Aside from being simple to use, the method is generally recognized for its accuracy and its ability to deal with small sample sizes, high-dimensional feature spaces and complex data structures. The random forest methodology has been successfully involved in many practical problems, including air quality prediction (winning code of the EMC data science global hackathon in 2012, see http://www.kaggle.com/c/dsg-hackathon), chemoinformatics [Svetnik et al. (2003)], ecology [Prasad, Iverson and Liaw (2006), Cutler et al. (2007)], 3D