# What's the Harm? Sharp Bounds on the Fraction Negatively Affected by Treatment

arXiv: https://arxiv.org/abs/2205.10327 · HTML: https://mlsanity.com/paper/2205.10327/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 74 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 22 of 22)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination](https://mlsanity.com/paper/1906.00285/) (2019)
- [Characterizing Fairness Over the Set of Good Models Under Selective Labels](https://mlsanity.com/paper/2101.00352/) (2021)
- [Interval Estimation of Individual-Level Causal Effects Under Unobserved Confounding](https://mlsanity.com/paper/1810.02894/) (2018)
- [Residual Unfairness in Fair Machine Learning from Prejudiced Data](https://mlsanity.com/paper/1806.02887/) (2018)
- [Characterizing Intersectional Group Fairness with Worst-Case Comparisons](https://mlsanity.com/paper/2101.01673/) (2021)
- [Bounds on the conditional and average treatment effect with unobserved confounding factors](https://mlsanity.com/paper/1808.09521/) (2018)
- [Fairness Evaluation in Presence of Biased Noisy Labels](https://mlsanity.com/paper/2003.13808/) (2020)
- [Doubly-Valid/Doubly-Sharp Sensitivity Analysis for Causal Inference with Unmeasured Confounding](https://mlsanity.com/paper/2112.11449/) (2021)
- [Sensitivity analysis via the proportion of unmeasured confounding](https://mlsanity.com/paper/1912.02793/) (2019)
- [Treatment Effect Risk: Bounds and Inference](https://mlsanity.com/paper/2201.05893/) (2022)
- [Decomposing Treatment Effect Variation](https://mlsanity.com/paper/1605.06566/) (2016)
- [Debiased Machine Learning of Set-Identified Linear Models](https://mlsanity.com/paper/1712.10024/) (2017)
- [Assessing Disparate Impacts of Personalized Interventions: Identifiability and Bounds](https://mlsanity.com/paper/1906.01552/) (2019)

## Built on by (most cited first, 15 of 15)

- [Inference on Strongly Identified Functionals of Weakly Identified Functions](https://mlsanity.com/paper/2208.08291/) (2022)
- [Quantifying Aleatoric Uncertainty of the Treatment Effect: A Novel Orthogonal Learner](https://mlsanity.com/paper/2411.03387/) (2024)
- [Policy Learning for Balancing Short-Term and Long-Term Rewards](https://mlsanity.com/paper/2405.03329/) (2024)
- [Policy Learning with Distributional Welfare](https://mlsanity.com/paper/2311.15878/) (2023)
- [Bayesian Safe Policy Learning with Chance Constrained Optimization: Application to Military Security Assessment during the Vietnam War](https://mlsanity.com/paper/2307.08840/) (2023)
- [Weak Supervision Performance Evaluation via Partial Identification](https://mlsanity.com/paper/2312.04601/) (2023)
- [Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes](https://mlsanity.com/paper/2404.00099/) (2024)
- [Algorithmic Fairness: A Tolerance Perspective](https://mlsanity.com/paper/2405.09543/) (2024)
- [Statistical Decision Theory with Counterfactual Loss](https://mlsanity.com/paper/2505.08908/) (2025)
- [Limits of Approximating the Median Treatment Effect](https://mlsanity.com/paper/2403.10618/) (2024)
- [Masking Causality and Conditional Dependence](https://mlsanity.com/paper/2603.06984/) (2026)
- [Causal EpiNets: Precision-corrected Bounds on Individual Treatment Effects using Epistemic Neural Networks](https://mlsanity.com/paper/2605.07065/) (2026)
- [Counterfactually Safe Reinforcement Learning](https://mlsanity.com/paper/2605.25114/) (2026)
- [Path-specific harm decomposition: A partial identification framework](https://mlsanity.com/paper/2609.29938/) (2026)
- [Same Predictions, Different Harms: Causal Auditing of Patient World Models](https://mlsanity.com/paper/2610.05198/) (2026)

## Abstract (arXiv)

The fundamental problem of causal inference -- that we never observe counterfactuals -- prevents us from identifying how many might be negatively affected by a proposed intervention. If, in an A/B test, half of users click (or buy, or watch, or renew, etc.), whether exposed to the standard experience A or a new one B, hypothetically it could be because the change affects no one, because the change positively affects half the user population to go from no-click to click while negatively affecting the other half, or something in between. While unknowable, this impact is clearly of material importance to the decision to implement a change or not, whether due to fairness, long-term, systemic, or operational considerations. We therefore derive the tightest-possible (i.e., sharp) bounds on the fraction negatively affected (and other related estimands) given data with only factual observations, whether experimental or observational. Naturally, the more we can stratify individuals by observable covariates, the tighter the sharp bounds. Since these bounds involve unknown functions that must be learned from data, we develop a robust inference algorithm that is efficient almost regardless of how and how fast these functions are learned, remains consistent when some are mislearned, and still gives valid conservative bounds when most are mislearned. Our methodology altogether therefore strongly supports credible conclusions: it avoids spuriously point-identifying this unknowable impact, focusing on the best bounds instead, and it permits exceedingly robust inference on these. We demonstrate our method in simulation studies and in a case study of career counseling for the unemployed.