# Generalised Boosted Forests

arXiv: https://arxiv.org/abs/2102.12561 · HTML: https://mlsanity.com/paper/2102.12561/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 18 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Builds on (most cited first, 4 of 4)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R](https://mlsanity.com/paper/1508.04409/) (2015)
- [Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife](https://mlsanity.com/paper/1311.4555/) (2013)
- [Boosting Random Forests to Reduce Bias; One-Step Boosted Forest and its Variance Estimate](https://mlsanity.com/paper/1803.08000/) (2018)

## Built on by (most cited first, 2 of 2)

- [The Infinitesimal Jackknife and Combinations of Models](https://mlsanity.com/paper/2209.00147/) (2022)
- [WildWood: a new Random Forest algorithm](https://mlsanity.com/paper/2109.08010/) (2021)

## Abstract (arXiv)

This paper extends recent work on boosting random forests to model non-Gaussian responses. Given an exponential family mathbbE[Y|X] = g-1(f(X)) our goal is to obtain an estimate for f. We start with an MLE-type estimate in the link space and then define generalised residuals from it. We use these residuals and some corresponding weights to fit a base random forest and then repeat the same to obtain a boost random forest. We call the sum of these three estimators a textitgeneralised boosted forest. We show with simulated and real data that both the random forest steps reduces test-set log-likelihood, which we treat as our primary metric. We also provide a variance estimator, which we can obtain with the same computational cost as the original estimate itself. Empirical experiments on real-world data and simulations demonstrate that the methods can effectively reduce bias, and that confidence interval coverage is conservative in the bulk of the covariate distribution.