# A Comparison of Methods for Treatment Assignment with an Application to Playlist Generation

arXiv: https://arxiv.org/abs/2004.11532 · HTML: https://mlsanity.com/paper/2004.11532/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2020.

## Built on by (most cited first, 9 of 9)

- [Causal Decision Making and Causal Effect Estimation Are Not the Same... and Why It Matters](https://mlsanity.com/paper/2104.04103/) (2021)
- [Maximizing the Success Probability of Policy Allocations in Online Systems](https://mlsanity.com/paper/2312.16267/) (2023)
- [Incremental Profit per Conversion: a Response Transformation for Uplift Modeling in E-Commerce Promotions](https://mlsanity.com/paper/2306.13759/) (2023)
- [Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimators](https://mlsanity.com/paper/2402.18392/) (2024)
- [CSPI-MT: Calibrated Safe Policy Improvement with Multiple Testing for Threshold Policies](https://mlsanity.com/paper/2408.12004/) (2024)
- [Causal Ordering Without Effect Estimation: A Framework for Using Proxies in Treatment Prioritization](https://mlsanity.com/paper/2206.12532/) (2022)
- [Modeling Financial Products and their Supply Chains](https://mlsanity.com/paper/2102.02329/) (2021)
- [Causal Post-Processing of Predictive Models](https://mlsanity.com/paper/2406.09567/) (2024)
- [Annotation-Assisted Learning of Treatment Policies From Multimodal Electronic Health Records](https://mlsanity.com/paper/2507.20993/) (2025)

## Abstract (arXiv)

This study presents a systematic comparison of methods for individual treatment assignment, a general problem that arises in many applications and has received significant attention from economists, computer scientists, and social scientists. We group the various methods proposed in the literature into three general classes of algorithms (or metalearners): learning models to predict outcomes (the O-learner), learning models to predict causal effects (the E-learner), and learning models to predict optimal treatment assignments (the A-learner). We compare the metalearners in terms of (1) their level of generality and (2) the objective function they use to learn models from data; we then discuss the implications that these characteristics have for modeling and decision making. Notably, we demonstrate analytically and empirically that optimizing for the prediction of outcomes or causal effects is not the same as optimizing for treatment assignments, suggesting that in general the A-learner should lead to better treatment assignments than the other metalearners. We demonstrate the practical implications of our findings in the context of choosing, for each user, the best algorithm for playlist generation in order to optimize engagement. This is the first comparison of the three different metalearners on a real-world application at scale (based on more than half a billion individual treatment assignments). In addition to supporting our analytical findings, the results show how large A/B tests can provide substantial value for learning treatment assignment policies, rather than simply choosing the variant that performs best on average.