# Bootstrap Sampling Rate Greater than 1.0 May Improve Random Forest Performance

arXiv: https://arxiv.org/abs/2410.04297 · HTML: https://mlsanity.com/paper/2410.04297/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 5 Oct 2024.

## Builds on (most cited first, 17 of 17)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning](https://mlsanity.com/paper/2007.04074/) (2020)
- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [Hyperparameters and Tuning Strategies for Random Forest](https://mlsanity.com/paper/1804.03515/) (2018)
- [YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization](https://mlsanity.com/paper/2109.03670/) (2021)
- [Hyperparameters in Reinforcement Learning and How To Tune Them](https://mlsanity.com/paper/2306.01324/) (2023)
- [To tune or not to tune the number of trees in random forest?](https://mlsanity.com/paper/1705.05654/) (2017)
- [PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning](https://mlsanity.com/paper/2306.12370/) (2023)
- [Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale](https://mlsanity.com/paper/2201.06834/) (2022)
- [Consistency of Online Random Forests](https://mlsanity.com/paper/1302.4853/) (2013)
- [A meta-learning recommender system for hyperparameter tuning: predicting when tuning improves SVM classifiers](https://mlsanity.com/paper/1906.01684/) (2019)
- [SigOpt Mulch: An Intelligent System for AutoML of Gradient Boosted Trees](https://mlsanity.com/paper/2307.04849/) (2023)
- [Generalized Self-Adapting Particle Swarm Optimization algorithm with archive of samples](https://mlsanity.com/paper/2002.12485/) (2020)
- [HyperTuner: A Cross-Layer Multi-Objective Hyperparameter Auto-Tuning Framework for Data Analytic Services](https://mlsanity.com/paper/2304.10051/) (2023)

## Abstract (arXiv)

Random forests (RFs) utilize bootstrap sampling to generate individual training sets for each component tree by sampling with replacement, with the sample size typically equal to that of the original training set (N). Previous research indicates that drawing fewer than N observations can also yield satisfactory results. The ratio of the number of observations in each bootstrap sample to the total number of training instances is referred to as the bootstrap rate (BR). Sampling more than N observations (BR > 1.0) has been explored only to a limited extent and has generally been considered ineffective. In this paper, we revisit this setup using 36 diverse datasets, evaluating BR values ranging from 1.2 to 5.0. Contrary to previous findings, we show that higher BR values can lead to statistically significant improvements in classification accuracy compared to standard settings (BR ≤ 1.0). Furthermore, we analyze how BR affects the leaf structure of decision trees within the RF and investigate factors influencing the optimal BR. Our results indicate that the optimal BR is primarily determined by the characteristics of the data set rather than the RF hyperparameters.