# A Large Scale Benchmark for Individual Treatment Effect Prediction and Uplift Modeling

arXiv: https://arxiv.org/abs/2111.10106 · HTML: https://mlsanity.com/paper/2111.10106/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 61 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition](https://mlsanity.com/paper/1707.02641/) (2017)
- [CausalML: Python Package for Causal Machine Learning](https://mlsanity.com/paper/2002.11631/) (2020)
- [Causal Decision Making and Causal Effect Estimation Are Not the Same... and Why It Matters](https://mlsanity.com/paper/2104.04103/) (2021)
- [Attribution Modeling Increases Efficiency of Bidding in Display Advertising](https://mlsanity.com/paper/1707.06409/) (2017)
- [Learning to rank for uplift modeling](https://mlsanity.com/paper/2002.05897/) (2020)
- [Large-scale Validation of Counterfactual Learning Methods: A Test-Bed](https://mlsanity.com/paper/1612.00367/) (2016)
- [Treatment Targeting by AUUC Maximization with Generalization Guarantees](https://mlsanity.com/paper/2012.09897/) (2020)
- [Doing Great at Estimating CATE? On the Neglected Assumptions in Benchmark Comparisons of Treatment Effect Estimators](https://mlsanity.com/paper/2107.13346/) (2021)
- [The Comparison of Methods for Individual Treatment Effect Detection](https://mlsanity.com/paper/1912.01443/) (2019)

## Built on by (most cited first, 12 of 12)

- [Applied Causal Inference Powered by ML and AI](https://mlsanity.com/paper/2403.02467/) (2024)
- [Explicit Feature Interaction-aware Uplift Network for Online Marketing](https://mlsanity.com/paper/2306.00315/) (2023)
- [Maximizing the Success Probability of Policy Allocations in Online Systems](https://mlsanity.com/paper/2312.16267/) (2023)
- [Rank-Learner: Orthogonal Ranking of Treatment Effects](https://mlsanity.com/paper/2602.03517/) (2026)
- [Privacy-Robust Incrementality Measurement for Advertising Systems under Signal Loss](https://mlsanity.com/paper/2606.03878/) (2026)
- [Improving uplift model evaluation on RCT data](https://mlsanity.com/paper/2210.02152/) (2022)
- [KDSM: An uplift modeling framework based on knowledge distillation and sample matching](https://mlsanity.com/paper/2303.02980/) (2023)
- [The impact of heteroskedasticity on uplift modeling](https://mlsanity.com/paper/2312.05234/) (2023)
- [PROXIMA: A Reliability Scoring Framework for Proxy Metrics in Online Controlled Experiments](https://mlsanity.com/paper/2604.14352/) (2026)
- [Real vs. Semi-Simulated: Rethinking Evaluation for Treatment Effect Estimation](https://mlsanity.com/paper/2605.10430/) (2026)
- [Doubly Robust Estimation of Causal Effect on CVR with Targeted Regularization](https://mlsanity.com/paper/2608.13461/) (2026)
- [When Does a Skill Add Value? Task-Conditional Gain Prediction for Selective Skill Use](https://mlsanity.com/paper/2609.32274/) (2026)

## Abstract (arXiv)

Individual Treatment Effect (ITE) prediction is an important area of research in machine learning which aims at explaining and estimating the causal impact of an action at the granular level. It represents a problem of growing interest in multiple sectors of application such as healthcare, online advertising or socioeconomics. To foster research on this topic we release a publicly available collection of 13.9 million samples collected from several randomized control trials, scaling up previously available datasets by a healthy 210x factor. We provide details on the data collection and perform sanity checks to validate the use of this data for causal inference tasks. First, we formalize the task of uplift modeling (UM) that can be performed with this data, along with the relevant evaluation metrics. Then, we propose synthetic response surfaces and heterogeneous treatment assignment providing a general set-up for ITE prediction. Finally, we report experiments to validate key characteristics of the dataset leveraging its size to evaluate and compare - with high statistical significance - a selection of baseline UM and ITE prediction methods.