# Estimation and Inference with Trees and Forests in High Dimensions

arXiv: https://arxiv.org/abs/2007.03210 · HTML: https://mlsanity.com/paper/2007.03210/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2020.

## Built on by (most cited first, 14 of 14)

- [Applied Causal Inference Powered by ML and AI](https://mlsanity.com/paper/2403.02467/) (2024)
- [Large Scale Prediction with Decision Trees](https://mlsanity.com/paper/2104.13881/) (2021)
- [Synthetic Combinations: A Causal Inference Framework for Combinatorial Interventions](https://mlsanity.com/paper/2303.14226/) (2023)
- [A cautionary tale on fitting decision trees to data from additive models: generalization lower bounds](https://mlsanity.com/paper/2110.09626/) (2021)
- [Fast Instrument Learning with Faster Rates](https://mlsanity.com/paper/2205.10772/) (2022)
- [Structure-agnostic Optimality of Doubly Robust Learning for Treatment Effect Estimation](https://mlsanity.com/paper/2402.14264/) (2024)
- [Label Ranking through Nonparametric Regression](https://mlsanity.com/paper/2111.02749/) (2021)
- [FACT: High-Dimensional Random Forests Inference](https://mlsanity.com/paper/2207.01678/) (2022)
- [Statistical Advantages of Oblique Randomized Decision Trees and Forests](https://mlsanity.com/paper/2407.02458/) (2024)
- [Revisiting Randomization in Greedy Model Search](https://mlsanity.com/paper/2506.15643/) (2025)
- [Orthogonal Series Estimation for the Ratio of Conditional Expectation Functions](https://mlsanity.com/paper/2212.13145/) (2022)
- [Causal Machine Learning for Moderation Effects](https://mlsanity.com/paper/2401.08290/) (2024)
- [Reproducibility and Geometric Intrinsic Dimensionality: An Investigation on Graph Neural Network Research](https://mlsanity.com/paper/2403.08438/) (2024)
- [Pure interaction effects unseen by Random Forests](https://mlsanity.com/paper/2406.15500/) (2024)

## Abstract (arXiv)

We analyze the finite sample mean squared error (MSE) performance of regression trees and forests in the high dimensional regime with binary features, under a sparsity constraint. We prove that if only r of the d features are relevant for the mean outcome function, then shallow trees built greedily via the CART empirical MSE criterion achieve MSE rates that depend only logarithmically on the ambient dimension d. We prove upper bounds, whose exact dependence on the number relevant variables r depends on the correlation among the features and on the degree of relevance. For strongly relevant features, we also show that fully grown honest forests achieve fast MSE rates and their predictions are also asymptotically normal, enabling asymptotically valid inference that adapts to the sparsity of the regression function.