# On Variance Estimation of Random Forests with Infinite-Order U-statistics

arXiv: https://arxiv.org/abs/2202.09008 · HTML: https://mlsanity.com/paper/2202.09008/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 30 references that are in the map.
- First posted on arXiv 18 Feb 2022.

## Builds on (most cited first, 6 of 6)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife](https://mlsanity.com/paper/1311.4555/) (2013)
- [Asymptotic Distributions and Rates of Convergence for Random Forests via Generalized U-statistics](https://mlsanity.com/paper/1905.10651/) (2019)
- [Randomized incomplete U-statistics in high dimensions](https://mlsanity.com/paper/1712.00771/) (2017)
- [Approximating high-dimensional infinite-order U-statistics: statistical and computational guarantees](https://mlsanity.com/paper/1901.01163/) (2019)

## Built on by (most cited first, 6 of 6)

- [Jackknife Variance Estimation for H'ajek-Dominated Generalized U-Statistics](https://mlsanity.com/paper/2509.12356/) (2025)
- [Random Forests as Statistical Procedures: Design, Variance, and Dependence](https://mlsanity.com/paper/2602.13104/) (2026)
- [Improving Random Forests by Smoothing](https://mlsanity.com/paper/2505.06852/) (2025)
- [Empirical Likelihood for Random Forests and Ensembles](https://mlsanity.com/paper/2511.13934/) (2025)
- [Predicting Mortgage Default with Machine Learning: AutoML, Class Imbalance, and Leakage Control](https://mlsanity.com/paper/2602.00120/) (2026)
- [Exact Finite-Sample Variance Decomposition of Subagging: A Spectral Filtering Perspective](https://mlsanity.com/paper/2604.10469/) (2026)

## Abstract (arXiv)

Infinite-order U-statistics (IOUS) has been used extensively on subbagging ensemble learning algorithms such as random forests to quantify its uncertainty. While normality results of IOUS have been studied extensively, its variance estimation approaches and theoretical properties remain mostly unexplored. Existing approaches mainly utilize the leading term dominance property in the Hoeffding decomposition. However, such a view usually leads to biased estimation when the kernel size is large or the sample size is small. On the other hand, while several unbiased estimators exist in the literature, their relationships and theoretical properties, especially the ratio consistency, have never been studied. These limitations lead to unguaranteed performances of constructed confidence intervals. To bridge these gaps in the literature, we propose a new view of the Hoeffding decomposition for variance estimation that leads to an unbiased estimator. Instead of leading term dominance, our view utilizes the dominance of the peak region. Moreover, we establish the connection and equivalence of our estimator with several existing unbiased variance estimators. Theoretically, we are the first to establish the ratio consistency of such a variance estimator, which justifies the coverage rate of confidence intervals constructed from random forests. Numerically, we further propose a local smoothing procedure to improve the estimator's finite sample performance. Extensive simulation studies show that our estimators enjoy lower bias and archive targeted coverage rates.