# Optimal Weighted Random Forests

arXiv: https://arxiv.org/abs/2305.10042 · HTML: https://mlsanity.com/paper/2305.10042/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 24 references that are in the map.
- First posted on arXiv 17 May 2023.

## Builds on (most cited first, 4 of 4)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Forecasting directional movements of stock prices for intraday trading using LSTM and random forests](https://mlsanity.com/paper/2004.10178/) (2020)

## Built on by (most cited first, 5 of 5)

- [When do Random Forests work?](https://mlsanity.com/paper/2504.12860/) (2025)
- [Binary Classification: Is Boosting stronger than Bagging?](https://mlsanity.com/paper/2410.19200/) (2024)
- [Adaptive Forests For Classification](https://mlsanity.com/paper/2510.22991/) (2025)
- [Mallows-type model averaging: Non-asymptotic analysis and all-subset combination](https://mlsanity.com/paper/2505.02637/) (2025)
- [PEARL: Performance-Enhanced Aggregated Representation Learning](https://mlsanity.com/paper/2509.24312/) (2025)

## Abstract (arXiv)

The random forest (RF) algorithm has become a very popular prediction method for its great flexibility and promising accuracy. In RF, it is conventional to put equal weights on all the base learners (trees) to aggregate their predictions. However, the predictive performances of different trees within the forest can be very different due to the randomization of the embedded bootstrap sampling and feature selection. In this paper, we focus on RF for regression and propose two optimal weighting algorithms, namely the 1 Step Optimal Weighted RF (1step-WRFopt) and 2 Steps Optimal Weighted RF (2steps-WRFopt), that combine the base learners through the weights determined by weight choice criteria. Under some regularity conditions, we show that these algorithms are asymptotically optimal in the sense that the resulting squared loss and risk are asymptotically identical to those of the infeasible but best possible model averaging estimator. Numerical studies conducted on real-world data sets indicate that these algorithms outperform the equal-weight forest and two other weighted RFs proposed in existing literature in most cases.