# Exogenous Randomness Empowering Random Forests

arXiv: https://arxiv.org/abs/2411.07554 · HTML: https://mlsanity.com/paper/2411.07554/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 12 Nov 2024.

## Builds on (most cited first, 8 of 8)

- [To tune or not to tune the number of trees in random forest?](https://mlsanity.com/paper/1705.05654/) (2017)
- [Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success](https://mlsanity.com/paper/1911.00190/) (2019)
- [Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers](https://mlsanity.com/paper/2402.01502/) (2024)
- [Inference with Mondrian Random Forests](https://mlsanity.com/paper/2310.09702/) (2023)
- [Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests](https://mlsanity.com/paper/2402.12668/) (2024)
- [Statistical-Computational Trade-offs for Recursive Adaptive Partitioning Estimators](https://mlsanity.com/paper/2411.04394/) (2024)
- [Analyzing CART](https://mlsanity.com/paper/1906.10086/) (2019)
- [Sharp Analysis of a Simple Model for Random Forests](https://mlsanity.com/paper/1805.02587/) (2018)

## Built on by (most cited first, 2 of 2)

- [Revisiting Randomization in Greedy Model Search](https://mlsanity.com/paper/2506.15643/) (2025)
- [CART Random Forests as Sequential Allocation over Random Opportunity Sets: A Stochastic-Control Theory of Ensemble Risk](https://mlsanity.com/paper/2605.26675/) (2026)

## Abstract (arXiv)

We offer theoretical and empirical insights into the impact of exogenous randomness on the effectiveness of random forests with tree-building rules independent of training data. We formally introduce the concept of exogenous randomness and identify two types of commonly existing randomness: Type I from feature subsampling, and Type II from tie-breaking in tree-building processes. We develop non-asymptotic expansions for the mean squared error (MSE) for both individual trees and forests and establish sufficient and necessary conditions for their consistency. In the special example of the linear regression model with independent features, our MSE expansions are more explicit, providing more understanding of the random forests' mechanisms. It also allows us to derive an upper bound on the MSE with explicit consistency rates for trees and forests. Guided by our theoretical findings, we conduct simulations to further explore how exogenous randomness enhances random forest performance. Our findings unveil that feature subsampling reduces both the bias and variance of random forests compared to individual trees, serving as an adaptive mechanism to balance bias and variance. Furthermore, our results reveal an intriguing phenomenon: the presence of noise features can act as a "blessing" in enhancing the performance of random forests thanks to feature subsampling.