# Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustness

arXiv: https://arxiv.org/abs/2603.20775 · HTML: https://mlsanity.com/paper/2603.20775/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 76 references that are in the map.
- First posted on arXiv 21 Mar 2026.

## Builds on (most cited first, 30 of 35)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithms](https://mlsanity.com/paper/2101.10943/) (2021)
- [On Inductive Biases for Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2106.03765/) (2021)
- [A unified survey of treatment effect heterogeneity modeling and uplift modeling](https://mlsanity.com/paper/2007.12769/) (2020)
- [Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions](https://mlsanity.com/paper/2007.12986/) (2020)
- [Optimal Transport for Treatment Effect Estimation](https://mlsanity.com/paper/2310.18286/) (2023)
- [Using Embeddings to Correct for Unobserved Confounding in Networks](https://mlsanity.com/paper/1902.04114/) (2019)
- [Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework](https://mlsanity.com/paper/2002.01711/) (2020)
- [In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2302.02923/) (2023)
- [Learning from History and Present: Next-item Recommendation via Discriminatively Exploiting User Behaviors](https://mlsanity.com/paper/1808.01075/) (2018)
- [Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation](https://mlsanity.com/paper/2211.01939/) (2022)
- [Explicit Feature Interaction-aware Uplift Network for Online Marketing](https://mlsanity.com/paper/2306.00315/) (2023)
- [A Survey on Causal Inference for Recommendation](https://mlsanity.com/paper/2303.11666/) (2023)
- [E-Commerce Promotions Personalization via Online Multiple-Choice Knapsack with Uplift Modeling](https://mlsanity.com/paper/2108.13298/) (2021)
- [Data Scarcity in Recommendation Systems: A Survey](https://mlsanity.com/paper/2312.10073/) (2023)
- [DESCN: Deep Entire Space Cross Networks for Individual Treatment Effect Estimation](https://mlsanity.com/paper/2207.09920/) (2022)
- [Anchored Causal Inference in the Presence of Measurement Error](https://mlsanity.com/paper/1906.00928/) (2019)
- [Conformal Counterfactual Inference under Hidden Confounding](https://mlsanity.com/paper/2405.12387/) (2024)
- [Mitigating Hidden Confounding Effects for Causal Recommendation](https://mlsanity.com/paper/2205.07499/) (2022)
- [Qini-based Uplift Regression](https://mlsanity.com/paper/1911.12474/) (2019)
- [Entire Chain Uplift Modeling with Context-Enhanced Learning for Intelligent Marketing](https://mlsanity.com/paper/2402.03379/) (2024)
- [The Causal Learning of Retail Delinquency](https://mlsanity.com/paper/2012.09448/) (2020)
- [Optimizing Treatment Allocation in the Presence of Interference](https://mlsanity.com/paper/2410.00075/) (2024)
- [Uplift Modeling Under Limited Supervision](https://mlsanity.com/paper/2403.19289/) (2024)

## Built on by (most cited first, 1 of 1)

- [UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation](https://mlsanity.com/paper/2608.00915/) (2026)

## Abstract (arXiv)

In personalized marketing, uplift models estimate the incremental effect of an intervention by modeling how customer behavior would change under alternative treatments using counterfactual analysis. However, real-world marketing data often exhibit various biases, such as selection bias, spillover effects, measurement error, and unobserved confounding. These biases can adversely affect both the accuracy of uplift estimation and the validity of evaluation metrics. Despite the importance of bias-aware assessment, there remains a lack of systematic studies evaluating how different models and metrics perform under such biased conditions. To bridge this gap, we design a systematic benchmarking framework. Unlike standard predictive tasks, real-world uplift datasets inherently lack counterfactual ground truth. This limitation renders the direct validation of evaluation metrics infeasible and prevents the precise quantification of biases. Therefore, a semi-synthetic approach serves as a critical enabler for systematic benchmarking. This approach effectively bridges the gap by retaining real-world feature dependencies while providing the ground truth needed to isolate structural biases. Our investigations reveal that (i) uplift targeting and prediction can manifest as distinct objectives, where proficiency in one does not ensure efficacy in the other; (ii) while many models exhibit inconsistent performance under diverse biases, TARNet shows notable robustness, providing insights for subsequent model design; (iii) the stability of evaluation metrics is linked to their mathematical alignment with the ATE, suggesting that ATE-approximating metrics yield more consistent model rankings under structural data imperfections. These findings suggest the need for more robust uplift models and evaluation metrics under real-world data imperfections.