# Domain constraints improve risk prediction when outcome data is missing

arXiv: https://arxiv.org/abs/2312.03878 · HTML: https://mlsanity.com/paper/2312.03878/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 119 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology](https://mlsanity.com/paper/1902.06543/) (2019)
- [A Fine-Grained Analysis on Distribution Shift](https://mlsanity.com/paper/2110.11328/) (2021)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [Extending the WILDS Benchmark for Unsupervised Adaptation](https://mlsanity.com/paper/2112.05090/) (2021)
- [Counterfactual Risk Assessments, Evaluation, and Fairness](https://mlsanity.com/paper/1909.00066/) (2019)
- [Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization](https://mlsanity.com/paper/2206.07837/) (2022)
- [Out-of-Domain Robustness via Targeted Augmentations](https://mlsanity.com/paper/2302.11861/) (2023)
- [Leveraging Expert Consistency to Improve Algorithmic Decision Support](https://mlsanity.com/paper/2101.09648/) (2021)
- [Fair Allocation through Selective Information Acquisition](https://mlsanity.com/paper/1911.02715/) (2019)
- [Coarse race data conceals disparities in clinical risk score performance](https://mlsanity.com/paper/2304.09270/) (2023)
- [Learning from a Biased Sample](https://mlsanity.com/paper/2209.01754/) (2022)
- [FADE: FAir Double Ensemble Learning for Observable and Counterfactual Outcomes](https://mlsanity.com/paper/2109.00173/) (2021)
- [Stay Ahead of Poachers: Illegal Wildlife Poaching Prediction and Patrol Planning Under Uncertainty with Field Test Evaluations](https://mlsanity.com/paper/1903.06669/) (2019)
- [Efficient Truncated Linear Regression with Unknown Noise Variance](https://mlsanity.com/paper/2208.12042/) (2022)
- [Counterfactual Prediction Under Outcome Measurement Error](https://mlsanity.com/paper/2302.11121/) (2023)
- [Detecting disparities in police deployments using dashcam data](https://mlsanity.com/paper/2305.15210/) (2023)
- [Fast Threshold Tests for Detecting Discrimination](https://mlsanity.com/paper/1702.08536/) (2017)
- [Learning the Truth From Only One Side of the Story](https://mlsanity.com/paper/2006.04858/) (2020)
- [Integrating Reward Maximization and Population Estimation: Sequential Decision-Making for Internal Revenue Service Audit Selection](https://mlsanity.com/paper/2204.11910/) (2022)
- [Algorithmic Censoring in Dynamic Learning Systems](https://mlsanity.com/paper/2305.09035/) (2023)
- [Trucks Don't Mean Trump: Diagnosing Human Error in Image Analysis](https://mlsanity.com/paper/2205.07333/) (2022)

## Built on by (most cited first, 8 of 8)

- [A Bayesian Spatial Model to Correct Under-Reporting in Urban Crowdsourcing](https://mlsanity.com/paper/2312.11754/) (2023)
- [Addressing Discretization-Induced Bias in Demographic Prediction](https://mlsanity.com/paper/2405.16762/) (2024)
- [From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions](https://mlsanity.com/paper/2406.18865/) (2024)
- [Predictive Performance Comparison of Decision Policies Under Confounding](https://mlsanity.com/paper/2404.00848/) (2024)
- [Bayesian Modeling of Zero-Shot Classifications for Urban Flood Detection](https://mlsanity.com/paper/2503.14754/) (2025)
- [Generative AI in Medicine](https://mlsanity.com/paper/2412.10337/) (2024)
- [Learning Disease Progression Models That Capture Health Disparities](https://mlsanity.com/paper/2412.16406/) (2024)
- [Data Fusion for High-Resolution Estimation](https://mlsanity.com/paper/2508.14858/) (2025)

## Abstract (arXiv)

Machine learning models are often trained to predict the outcome resulting from a human decision. For example, if a doctor decides to test a patient for disease, will the patient test positive? A challenge is that historical decision-making determines whether the outcome is observed: we only observe test outcomes for patients doctors historically tested. Untested patients, for whom outcomes are unobserved, may differ from tested patients along observed and unobserved dimensions. We propose a Bayesian model class which captures this setting. The purpose of the model is to accurately estimate risk for both tested and untested patients. Estimating this model is challenging due to the wide range of possibilities for untested patients. To address this, we propose two domain constraints which are plausible in health settings: a prevalence constraint, where the overall disease prevalence is known, and an expertise constraint, where the human decision-maker deviates from purely risk-based decision-making only along a constrained feature set. We show theoretically and on synthetic data that domain constraints improve parameter inference. We apply our model to a case study of cancer risk prediction, showing that the model's inferred risk predicts cancer diagnoses, its inferred testing policy captures known public health policies, and it can identify suboptimalities in test allocation. Though our case study is in healthcare, our analysis reveals a general class of domain constraints which can improve model estimation in many settings.