# Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects arXiv: https://arxiv.org/abs/2111.07966 ยท HTML: https://mlsanity.com/paper/2111.07966/ ## Key facts - Cited by 31 papers in the map (data through 14 Oct 2026). - Builds on 8 of its 92 references that are in the map. - First posted on arXiv 15 Nov 2021. ## Builds on (most cited first, 8 of 8) - [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015) - [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017) - [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017) - [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016) - [Machine Learning Methods Economists Should Know About](https://mlsanity.com/paper/1903.10075/) (2019) - [Estimating heterogeneous treatment effects with right-censored data via causal survival forests](https://mlsanity.com/paper/2001.09887/) (2020) - [Experimental Evaluation of Individualized Treatment Rules](https://mlsanity.com/paper/1905.05389/) (2019) - [Treatment Allocation under Uncertain Costs](https://mlsanity.com/paper/2103.11066/) (2021) ## Built on by (most cited first, 30 of 31) - [Diagnosing Model Performance Under Distribution Shift](https://mlsanity.com/paper/2303.02011/) (2023) - [Zero-shot causal learning](https://mlsanity.com/paper/2301.12292/) (2023) - [Causal isotonic calibration for heterogeneous treatment effects](https://mlsanity.com/paper/2302.14011/) (2023) - [Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding](https://mlsanity.com/paper/2212.09844/) (2022) - [Machine Learning Who to Nudge: Causal vs Predictive Targeting in a Field Experiment on Student Financial Aid Renewal](https://mlsanity.com/paper/2310.08672/) (2023) - [Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments](https://mlsanity.com/paper/2203.14511/) (2022) - [Proximal Causal Learning of Conditional Average Treatment Effects](https://mlsanity.com/paper/2301.10913/) (2023) - [Calibration Error for Heterogeneous Treatment Effects](https://mlsanity.com/paper/2203.13364/) (2022) - [Evaluating the Effectiveness of Index-Based Treatment Allocation](https://mlsanity.com/paper/2402.11771/) (2024) - [Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression](https://mlsanity.com/paper/2207.09139/) (2022) - [Optimizing Multi-Scale Representations to Detect Effect Heterogeneity Using Earth Observation and Computer Vision: Applications to Two Anti-Poverty RCTs](https://mlsanity.com/paper/2411.02134/) (2024) - [Comparing Targeting Strategies for Maximizing Social Welfare with Limited Resources](https://mlsanity.com/paper/2411.07414/) (2024) - [Rank-Learner: Orthogonal Ranking of Treatment Effects](https://mlsanity.com/paper/2602.03517/) (2026) - [CTRL Your Shift: Clustered Transfer Residual Learning for Many Small Datasets](https://mlsanity.com/paper/2508.11144/) (2025) - [Federated Learning of Quantile Inference under Local Differential Privacy](https://mlsanity.com/paper/2509.21800/) (2025) - [Fostering the Ecosystem of AI for Social Impact Requires Expanding and Strengthening Evaluation Standards](https://mlsanity.com/paper/2510.18238/) (2025) - [A Statistical Test for the Benefits of Personalizing Interventions](https://mlsanity.com/paper/2607.08951/) (2026) - [UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation](https://mlsanity.com/paper/2608.00915/) (2026) - [Improving uplift model evaluation on RCT data](https://mlsanity.com/paper/2210.02152/) (2022) - [Statistical Performance Guarantee for Subgroup Identification with Generic Machine Learning](https://mlsanity.com/paper/2310.07973/) (2023) - [Neyman Meets Causal Machine Learning: Experimental Evaluation of Individualized Treatment Rules](https://mlsanity.com/paper/2404.17019/) (2024) - [Causal Post-Processing of Predictive Models](https://mlsanity.com/paper/2406.09567/) (2024) - [Qini Curve Estimation under Clustered Network Interference](https://mlsanity.com/paper/2502.20097/) (2025) - [Statistical Learning for Heterogeneous Treatment Effects: Pretraining, Prognosis, and Prediction](https://mlsanity.com/paper/2505.00310/) (2025) - [Annotation-Assisted Learning of Treatment Policies From Multimodal Electronic Health Records](https://mlsanity.com/paper/2507.20993/) (2025) - [Learning to Optimize Feedback for One Million Students: Insights from Multi-Armed and Contextual Bandits in Large-Scale Online Tutoring](https://mlsanity.com/paper/2508.00270/) (2025) - [Real vs. Semi-Simulated: Rethinking Evaluation for Treatment Effect Estimation](https://mlsanity.com/paper/2605.10430/) (2026) - [Detecting an Effect Is Not Learning to Act on It: A Reward-SNR Floor for LLM Acquisition Agents](https://mlsanity.com/paper/2608.10441/) (2026) - [A Multimodal Foundation Model for Longitudinal Patient Representation and Scalable Insight Generation in Oncology](https://mlsanity.com/paper/2608.24688/) (2026) - [Cross-Fitting Under Nonregularity: Normality and Inference via Locality](https://mlsanity.com/paper/2610.02944/) (2026) ## Abstract (arXiv) There are a number of available methods for selecting whom to prioritize for treatment, including ones based on treatment effect estimation, risk scoring, and hand-crafted rules. We propose rank-weighted average treatment effect (RATE) metrics as a simple and general family of metrics for comparing and testing the quality of treatment prioritization rules. RATE metrics are agnostic as to how the prioritization rules were derived, and only assess how well they identify individuals that benefit the most from treatment. We define a family of RATE estimators and prove a central limit theorem that enables asymptotically exact inference in a wide variety of randomized and observational study settings. RATE metrics subsume a number of existing metrics, including the Qini coefficient, and our analysis directly yields inference methods for these metrics. We showcase RATE in the context of a number of applications, including optimal targeting of aspirin to stroke patients.