Neyman Meets Causal Machine Learning: Experimental Evaluation of Individualized Treatment Rules
Not on the map. Why?
Abstract (arXiv)
A century ago, Neyman showed how to evaluate the efficacy of treatment using a randomized experiment under a minimal set of assumptions. This classical repeated sampling framework serves as a basis of routine experimental analyses conducted by today's scientists across disciplines. In this paper, we demonstrate that Neyman's methodology can also be used to experimentally evaluate the efficacy of individualized treatment rules (ITRs), which are derived by modern causal machine learning algorithms. In particular, we show how to account for additional uncertainty resulting from a training process based on cross-fitting. The primary advantage of Neyman's approach is that it can be applied to any ITR regardless of the properties of machine learning algorithms that are used to derive the ITR. We also show, somewhat surprisingly, that for certain metrics, it is more efficient to conduct this ex-post experimental evaluation of an ITR than to conduct an ex-ante experimental evaluation that randomly assigns some units to the ITR. Our analysis demonstrates that Neyman's repeated sampling framework is as relevant for causal inference today as it has been since its inception.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 30 references that are in the map. First posted on arXiv 25 Apr 2024.
Builds on
12 of 30 references.
References · 12
- 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 597
- 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
- 2015Recursive Partitioning for Heterogeneous Causal Effectscited by 254
- 2017Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competitioncited by 106
- 2017Balanced Policy Evaluation and Learningcited by 51
- 2017Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effectscited by 41
- 2015Residual Weighted Learning for Estimating Individualized Treatment Rulescited by 37
- 2021Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effectscited by 29
- 2019Experimental Evaluation of Individualized Treatment Rulescited by 16
- 2021Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessmentcited by 10
- 2022Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experimentscited by 7
- 2021Efficient Learning of Optimal Individualized Treatment Rules for Heteroscedastic or Misspecified Treatment-Free Effect Modelscited by 2
Built on by
Not cited yet.