# Regression-Enhanced Random Forests

arXiv: https://arxiv.org/abs/1904.10416 · HTML: https://mlsanity.com/paper/1904.10416/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2019.

## Built on by (most cited first, 5 of 5)

- [Hybrid additive modeling with partial dependence for supervised regression and dynamical systems forecasting](https://mlsanity.com/paper/2307.02229/) (2023)
- [A comprehensive GeoAI review: Progress, Challenges and Outlooks](https://mlsanity.com/paper/2412.11643/) (2024)
- [Towards Accurate Forecasting of Renewable Energy : Building Datasets and Benchmarking Machine Learning Models for Solar and Wind Power in France](https://mlsanity.com/paper/2504.16100/) (2025)
- [TMPNN: High-Order Polynomial Regression Based on Taylor Map Factorization](https://mlsanity.com/paper/2307.16105/) (2023)
- [Integrating Transformations in Probabilistic Circuits](https://mlsanity.com/paper/2310.04354/) (2023)

## Abstract (arXiv)

Random forest (RF) methodology is one of the most popular machine learning techniques for prediction problems. In this article, we discuss some cases where random forests may suffer and propose a novel generalized RF method, namely regression-enhanced random forests (RERFs), that can improve on RFs by borrowing the strength of penalized parametric regression. The algorithm for constructing RERFs and selecting its tuning parameters is described. Both simulation study and real data examples show that RERFs have better predictive performance than RFs in important situations often encountered in practice. Moreover, RERFs may incorporate known relationships between the response and the predictors, and may give reliable predictions in extrapolation problems where predictions are required at points out of the domain of the training dataset. Strategies analogous to those described here can be used to improve other machine learning methods via combination with penalized parametric regression techniques.