# Treatment Effect Risk: Bounds and Inference

arXiv: https://arxiv.org/abs/2201.05893 · HTML: https://mlsanity.com/paper/2201.05893/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 67 references that are in the map.
- First posted on arXiv 15 Jan 2022.

## Builds on (most cited first, 25 of 25)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination](https://mlsanity.com/paper/1906.00285/) (2019)
- [Balanced Policy Evaluation and Learning](https://mlsanity.com/paper/1705.07384/) (2017)
- [Residual Unfairness in Fair Machine Learning from Prejudiced Data](https://mlsanity.com/paper/1806.02887/) (2018)
- [Experimental Design in Two-Sided Platforms: An Analysis of Bias](https://mlsanity.com/paper/2002.05670/) (2020)
- [The Sensitivity of Counterfactual Fairness to Unmeasured Confounding](https://mlsanity.com/paper/1907.01040/) (2019)
- [Program Evaluation and Causal Inference with High-Dimensional Data](https://mlsanity.com/paper/1311.2645/) (2013)
- [Recursive Partitioning for Personalization using Observational Data](https://mlsanity.com/paper/1608.08925/) (2016)
- [Doubly-Valid/Doubly-Sharp Sensitivity Analysis for Causal Inference with Unmeasured Confounding](https://mlsanity.com/paper/2112.11449/) (2021)
- [Robust Sample Average Approximation](https://mlsanity.com/paper/1408.4445/) (2014)
- [Localized Debiased Machine Learning: Efficient Inference on Quantile Treatment Effects and Beyond](https://mlsanity.com/paper/1912.12945/) (2019)
- [Policy Learning with Observational Data](https://mlsanity.com/paper/1702.02896/) (2017)
- [Randomization Inference for Treatment Effect Variation](https://mlsanity.com/paper/1412.5000/) (2014)
- [Decomposing Treatment Effect Variation](https://mlsanity.com/paper/1605.06566/) (2016)
- [Evaluating the Impact of Treating the Optimal Subgroup](https://mlsanity.com/paper/1603.06372/) (2016)
- [Assessing Disparate Impacts of Personalized Interventions: Identifiability and Bounds](https://mlsanity.com/paper/1906.01552/) (2019)
- [Rejoinder: New Objectives for Policy Learning](https://mlsanity.com/paper/2012.03130/) (2020)

## Built on by (most cited first, 12 of 12)

- [Applied Causal Inference Powered by ML and AI](https://mlsanity.com/paper/2403.02467/) (2024)
- [B-Learner: Quasi-Oracle Bounds on Heterogeneous Causal Effects Under Hidden Confounding](https://mlsanity.com/paper/2304.10577/) (2023)
- [Bounds on Representation-Induced Confounding Bias for Treatment Effect Estimation](https://mlsanity.com/paper/2311.11321/) (2023)
- [Robust and Agnostic Learning of Conditional Distributional Treatment Effects](https://mlsanity.com/paper/2205.11486/) (2022)
- [What's the Harm? Sharp Bounds on the Fraction Negatively Affected by Treatment](https://mlsanity.com/paper/2205.10327/) (2022)
- [Partial Identification of Dose Responses with Hidden Confounders](https://mlsanity.com/paper/2204.11206/) (2022)
- [Ensembled Prediction Intervals for Causal Outcomes Under Hidden Confounding](https://mlsanity.com/paper/2306.09520/) (2023)
- [Policy Learning for Balancing Short-Term and Long-Term Rewards](https://mlsanity.com/paper/2405.03329/) (2024)
- [Distributionally Robust Policy Learning under Concept Drifts](https://mlsanity.com/paper/2412.14297/) (2024)
- [Policy Learning with Distributional Welfare](https://mlsanity.com/paper/2311.15878/) (2023)
- [Treatment Effects in Extreme Regimes](https://mlsanity.com/paper/2306.11697/) (2023)
- [GAAVI: Global Asymptotic Anytime Valid Inference for the Conditional Mean Function](https://mlsanity.com/paper/2602.08096/) (2026)

## Abstract (arXiv)

Since the average treatment effect (ATE) measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10% of the population. Assessing such risk is difficult, however, because any one individual treatment effect (ITE) is never observed, so the 10% worst-affected cannot be identified, while distributional treatment effects only compare the first deciles within each treatment group, which does not correspond to any 10%-subpopulation. In this paper we consider how to nonetheless assess this important risk measure, formalized as the conditional value at risk (CVaR) of the ITE-distribution. We leverage the availability of pre-treatment covariates and characterize the tightest-possible upper and lower bounds on ITE-CVaR given by the covariate-conditional average treatment effect (CATE) function. We then proceed to study how to estimate these bounds efficiently from data and construct confidence intervals. This is challenging even in randomized experiments as it requires understanding the distribution of the unknown CATE function, which can be very complex if we use rich covariates so as to best control for heterogeneity. We develop a debiasing method that overcomes this and prove it enjoys favorable statistical properties even when CATE and other nuisances are estimated by black-box machine learning or even inconsistently. Studying a hypothetical change to French job-search counseling services, our bounds and inference demonstrate a small social benefit entails a negative impact on a substantial subpopulation.