# Experimental Evaluation of Individualized Treatment Rules

arXiv: https://arxiv.org/abs/1905.05389 · HTML: https://mlsanity.com/paper/1905.05389/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2019.

## Built on by (most cited first, 16 of 16)

- [Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects](https://mlsanity.com/paper/2111.07966/) (2021)
- [A Survey of Deep Causal Models and Their Industrial Applications](https://mlsanity.com/paper/2209.08860/) (2022)
- [Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments](https://mlsanity.com/paper/2203.14511/) (2022)
- [Individualized Policy Evaluation and Learning under Clustered Network Interference](https://mlsanity.com/paper/2311.02467/) (2023)
- [Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation](https://mlsanity.com/paper/2302.02570/) (2023)
- [A Statistical Test for the Benefits of Personalizing Interventions](https://mlsanity.com/paper/2607.08951/) (2026)
- [Using Machine Learning to Test Causal Hypotheses in Conjoint Analysis](https://mlsanity.com/paper/2201.08343/) (2022)
- [Overview and practical recommendations on using Shapley Values for identifying predictive biomarkers via CATE modeling](https://mlsanity.com/paper/2505.01145/) (2025)
- [Federated Instrumental Variable Analysis via Federated Generalized Method of Moments](https://mlsanity.com/paper/2505.21012/) (2025)
- [Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning](https://mlsanity.com/paper/2310.07973/) (2023)
- [Neyman Meets Causal Machine Learning: Experimental Evaluation of Individualized Treatment Rules](https://mlsanity.com/paper/2404.17019/) (2024)
- [Qini Curve Estimation under Clustered Network Interference](https://mlsanity.com/paper/2502.20097/) (2025)
- [Statistical Learning for Heterogeneous Treatment Effects: Pretraining, Prognosis, and Prediction](https://mlsanity.com/paper/2505.00310/) (2025)
- [Deployment of AI-Assisted Interventions: Capacity Constraints and Noisy Compliance](https://mlsanity.com/paper/2604.14370/) (2026)
- [Learning Who to Treat When Treatment is Missing](https://mlsanity.com/paper/2607.14346/) (2026)
- [Detecting an Effect Is Not Learning to Act on It: A Reward-SNR Floor for LLM Acquisition Agents](https://mlsanity.com/paper/2608.10441/) (2026)

## Abstract (arXiv)

The increasing availability of individual-level data has led to numerous applications of individualized (or personalized) treatment rules (ITRs). Policy makers often wish to empirically evaluate ITRs and compare their relative performance before implementing them in a target population. We propose a new evaluation metric, the population average prescriptive effect (PAPE). The PAPE compares the performance of ITR with that of non-individualized treatment rule, which randomly treats the same proportion of units. Averaging the PAPE over a range of budget constraints yields our second evaluation metric, the area under the prescriptive effect curve (AUPEC). The AUPEC represents an overall performance measure for evaluation, like the area under the receiver and operating characteristic curve (AUROC) does for classification, and is a generalization of the QINI coefficient utilized in uplift modeling. We use Neyman's repeated sampling framework to estimate the PAPE and AUPEC and derive their exact finite-sample variances based on random sampling of units and random assignment of treatment. We extend our methodology to a common setting, in which the same experimental data is used to both estimate and evaluate ITRs. In this case, our variance calculation incorporates the additional uncertainty due to random splits of data used for cross-validation. The proposed evaluation metrics can be estimated without requiring modeling assumptions, asymptotic approximation, or resampling methods. As a result, it is applicable to any ITR including those based on complex machine learning algorithms. The open-source software package is available for implementing the proposed methodology.