# Honesty in Causal Forests: When It Helps and When It Hurts

arXiv: https://arxiv.org/abs/2506.13107 · HTML: https://mlsanity.com/paper/2506.13107/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 26 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 9 of 9)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition](https://mlsanity.com/paper/1707.02641/) (2017)
- [A comparison of methods for model selection when estimating individual treatment effects](https://mlsanity.com/paper/1804.05146/) (2018)
- [Stochastic Optimization Forests](https://mlsanity.com/paper/2008.07473/) (2020)
- [The Econometrics of Randomized Experiments](https://mlsanity.com/paper/1607.00698/) (2016)
- [Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments](https://mlsanity.com/paper/2203.14511/) (2022)
- [On the Pointwise Behavior of Recursive Partitioning and Its Implications for Heterogeneous Causal Effect Estimation](https://mlsanity.com/paper/2211.10805/) (2022)

## Abstract (arXiv)

Causal forests estimate how treatment effects vary across individuals, guiding personalized interventions in areas like marketing, operations, and public policy. A standard practice is honest estimation: dividing the data into two samples, one to define subgroups and another to estimate treatment effects within them. This is intended to reduce overfitting and is the default in many software packages. But is it the right choice? We show that honest estimation can reduce the accuracy of estimates of individual treatment effects, especially when effect heterogeneity is substantial and datasets are large enough to detect it. The reason is a bias-variance trade-off: honesty lowers the risk of overfitting but increases the risk of underfitting by limiting the data available to detect and model heterogeneity. Across more than 7,000 benchmark datasets, we find that the cost of using honesty by default can be as high as requiring 27% more data to match the performance of models trained without it. Honesty is best understood as a form of regularization. Whether to adopt it should depend on the goals of the application and its empirical performance, not on reflexive default use.