# Random Forests as Statistical Procedures: Design, Variance, and Dependence

arXiv: https://arxiv.org/abs/2602.13104 · HTML: https://mlsanity.com/paper/2602.13104/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Builds on (most cited first, 10 of 10)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success](https://mlsanity.com/paper/1911.00190/) (2019)
- [Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers](https://mlsanity.com/paper/2402.01502/) (2024)
- [Estimating the Algorithmic Variance of Randomized Ensembles via the Bootstrap](https://mlsanity.com/paper/1907.08742/) (2019)
- [Getting Better from Worse: Augmented Bagging and a Cautionary Tale of Variable Importance](https://mlsanity.com/paper/2003.03629/) (2020)
- [On Variance Estimation of Random Forests with Infinite-Order U-statistics](https://mlsanity.com/paper/2202.09008/) (2022)
- [Measuring the Algorithmic Convergence of Randomized Ensembles: The Regression Setting](https://mlsanity.com/paper/1908.01251/) (2019)

## Built on by (most cited first, 1 of 1)

- [Exact Finite-Sample Variance Decomposition of Subagging: A Spectral Filtering Perspective](https://mlsanity.com/paper/2604.10469/) (2026)

## Abstract (arXiv)

We develop a finite-sample, design-based theory for random forests in which each tree is a randomized conditional predictor acting on fixed covariates and the forest is their Monte Carlo average. An exact variance identity separates Monte Carlo error from a covariance floor that persists under infinite aggregation. The floor arises through two mechanisms: observation reuse, where the same training outcomes receive weight across multiple trees, and partition alignment, where independently generated trees discover similar conditional prediction rules. We prove the floor is strictly positive under minimal conditions and show that alignment persists even when sample splitting eliminates observation overlap entirely. We introduce procedure-aligned synthetic resampling (PASR) to estimate the covariance floor, decomposing the total prediction uncertainty of a deployed forest into interpretable components. For continuous outcomes, resulting prediction intervals achieve nominal coverage with a theoretically guaranteed conservative bias direction. For classification forests, the PASR estimator is asymptotically unbiased, providing the first pointwise confidence intervals for predicted conditional probabilities from a deployed forest. Nominal coverage is maintained across a range of design configurations for both outcome types, including high-dimensional settings. The underlying theory extends to any tree-based ensemble with an exchangeable tree-generating mechanism.