# Counterfactual Prediction Under Outcome Measurement Error

arXiv: https://arxiv.org/abs/2302.11121 · HTML: https://mlsanity.com/paper/2302.11121/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 83 references that are in the map.
- First posted on arXiv 22 Feb 2023.

## Builds on (most cited first, 30 of 31)

- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Performative Prediction](https://mlsanity.com/paper/2002.06673/) (2020)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Are Anchor Points Really Indispensable in Label-Noise Learning?](https://mlsanity.com/paper/1906.00189/) (2019)
- [Semiparametric doubly robust targeted double machine learning: a review](https://mlsanity.com/paper/2203.06469/) (2022)
- [A Survey of Label-noise Representation Learning: Past, Present and Future](https://mlsanity.com/paper/2011.04406/) (2020)
- [Learning with Symmetric Label Noise: The Importance of Being Unhinged](https://mlsanity.com/paper/1505.07634/) (2015)
- [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022)
- [Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise](https://mlsanity.com/paper/2012.05458/) (2020)
- [Double/Debiased/Neyman Machine Learning of Treatment Effects](https://mlsanity.com/paper/1701.08687/) (2017)
- [Generalization Bounds and Representation Learning for Estimation of Potential Outcomes and Causal Effects](https://mlsanity.com/paper/2001.07426/) (2020)
- [Classification with Asymmetric Label Noise: Consistency and Maximal Denoising](https://mlsanity.com/paper/1303.1208/) (2013)
- [Fair Classification with Group-Dependent Label Noise](https://mlsanity.com/paper/2011.00379/) (2020)
- [VCNet and Functional Targeted Regularization For Learning Causal Effects of Continuous Treatments](https://mlsanity.com/paper/2103.07861/) (2021)
- [Counterfactual Risk Assessments, Evaluation, and Fairness](https://mlsanity.com/paper/1909.00066/) (2019)
- [Residual Unfairness in Fair Machine Learning from Prejudiced Data](https://mlsanity.com/paper/1806.02887/) (2018)
- [Part-dependent Label Noise: Towards Instance-dependent Label Noise](https://mlsanity.com/paper/2006.07836/) (2020)
- [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022)
- [Fairness Evaluation in Presence of Biased Noisy Labels](https://mlsanity.com/paper/2003.13808/) (2020)
- [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022)
- [Unbiased Risk Estimators Can Mislead: A Case Study of Learning with Complementary Labels](https://mlsanity.com/paper/2007.02235/) (2020)
- [Counterfactual Predictions under Runtime Confounding](https://mlsanity.com/paper/2006.16916/) (2020)
- [Leveraging Expert Consistency to Improve Algorithmic Decision Support](https://mlsanity.com/paper/2101.09648/) (2021)
- [The effect of differential victim crime reporting on predictive policing systems](https://mlsanity.com/paper/2102.00128/) (2021)
- [Partial Identifiability in Discrete Data With Measurement Error](https://mlsanity.com/paper/2012.12449/) (2020)
- [Classification with unknown class-conditional label noise on non-compact feature spaces](https://mlsanity.com/paper/1902.05627/) (2019)
- [Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding](https://mlsanity.com/paper/2212.09844/) (2022)

## Built on by (most cited first, 10 of 10)

- [Domain constraints improve risk prediction when outcome data is missing](https://mlsanity.com/paper/2312.03878/) (2023)
- [A Bayesian Spatial Model to Correct Under-Reporting in Urban Crowdsourcing](https://mlsanity.com/paper/2312.11754/) (2023)
- ["This is not a data problem": Algorithms and Power in Public Higher Education in Canada](https://mlsanity.com/paper/2403.13969/) (2024)
- [The Neutrality Fallacy: When Algorithmic Fairness Interventions are (Not) Positive Action](https://mlsanity.com/paper/2404.12143/) (2024)
- [The Value of Prediction in Identifying the Worst-Off](https://mlsanity.com/paper/2501.19334/) (2025)
- [From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions](https://mlsanity.com/paper/2406.18865/) (2024)
- [Operationalizing Counterfactual Metrics: Incentives, Ranking, and Information Asymmetry](https://mlsanity.com/paper/2305.14595/) (2023)
- [Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Tasks](https://mlsanity.com/paper/2507.02819/) (2025)
- [Score Design for Multi-Criteria Incentivization](https://mlsanity.com/paper/2410.06290/) (2024)
- [Proxy-Guided Measurement Calibration](https://mlsanity.com/paper/2603.09288/) (2026)

## Abstract (arXiv)

Across domains such as medicine, employment, and criminal justice, predictive models often target labels that imperfectly reflect the outcomes of interest to experts and policymakers. For example, clinical risk assessments deployed to inform physician decision-making often predict measures of healthcare utilization (e.g., costs, hospitalization) as a proxy for patient medical need. These proxies can be subject to outcome measurement error when they systematically differ from the target outcome they are intended to measure. However, prior modeling efforts to characterize and mitigate outcome measurement error overlook the fact that the decision being informed by a model often serves as a risk-mitigating intervention that impacts the target outcome of interest and its recorded proxy. Thus, in these settings, addressing measurement error requires counterfactual modeling of treatment effects on outcomes. In this work, we study intersectional threats to model reliability introduced by outcome measurement error, treatment effects, and selection bias from historical decision-making policies. We develop an unbiased risk minimization method which, given knowledge of proxy measurement error properties, corrects for the combined effects of these challenges. We also develop a method for estimating treatment-dependent measurement error parameters when these are unknown in advance. We demonstrate the utility of our approach theoretically and via experiments on real-world data from randomized controlled trials conducted in healthcare and employment domains. As importantly, we demonstrate that models correcting for outcome measurement error or treatment effects alone suffer from considerable reliability limitations. Our work underscores the importance of considering intersectional threats to model validity during the design and evaluation of predictive models for decision support.