# Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests

arXiv: https://arxiv.org/abs/2402.12668 · HTML: https://mlsanity.com/paper/2402.12668/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 35 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 5 of 5)

- [Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers](https://mlsanity.com/paper/1504.07676/) (2015)
- [Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success](https://mlsanity.com/paper/1911.00190/) (2019)
- [Extended Comparisons of Best Subset Selection, Forward Stepwise Selection, and the Lasso](https://mlsanity.com/paper/1707.08692/) (2017)
- [Trees, Forests, Chickens, and Eggs: When and Why to Prune Trees in a Random Forest](https://mlsanity.com/paper/2103.16700/) (2021)
- [Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance](https://mlsanity.com/paper/2003.03629/) (2020)

## Built on by (most cited first, 6 of 6)

- [Exogenous Randomness Empowering Random Forests](https://mlsanity.com/paper/2411.07554/) (2024)
- [When do Random Forests work?](https://mlsanity.com/paper/2504.12860/) (2025)
- [Revisiting Randomization in Greedy Model Search](https://mlsanity.com/paper/2506.15643/) (2025)
- [A Novel Method to Determine Total Oxidant Concentration Produced by Non-Thermal Plasma Based on Image Processing and Machine Learning](https://mlsanity.com/paper/2509.00479/) (2025)
- [Distributional Random Forests for Complex Survey Designs](https://mlsanity.com/paper/2512.08179/) (2025)
- [CART Random Forests as Sequential Allocation over Random Opportunity Sets: A Stochastic-Control Theory of Ensemble Risk](https://mlsanity.com/paper/2605.26675/) (2026)

## Abstract (arXiv)

We study the often overlooked phenomenon, first noted in citebreiman2001random, that random forests appear to reduce bias compared to bagging. Motivated by an interesting paper by citementch2020randomization, where the authors explain the success of random forests in low signal-to-noise ratio (SNR) settings through regularization, we explore how random forests can capture patterns in the data that bagging ensembles fail to capture. We empirically demonstrate that in the presence of such patterns, random forests reduce bias along with variance and can increasingly outperform bagging ensembles when SNR is high. Our observations offer insights into the real-world success of random forests across a range of SNRs and enhance our understanding of the difference between random forests and bagging ensembles. Our investigations also yield practical insights into the importance of tuning mtry in random forests.