# A Practical Upper Bound on Selection Bias Effects in Medical Prediction Models

arXiv: https://arxiv.org/abs/2606.00563 · HTML: https://mlsanity.com/paper/2606.00563/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 110 references that are in the map.
- First posted on arXiv 30 May 2026.

## Builds on (most cited first, 17 of 17)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [Learning Likelihoods with Conditional Normalizing Flows](https://mlsanity.com/paper/1912.00042/) (2019)
- [A review of domain adaptation without target labels](https://mlsanity.com/paper/1901.05335/) (2019)
- [Fair Generative Modeling via Weak Supervision](https://mlsanity.com/paper/1910.12008/) (2019)
- [Bias in Large Language Models: Origin, Evaluation, and Mitigation](https://mlsanity.com/paper/2411.10915/) (2024)
- [f-Domain-Adversarial Learning: Theory and Algorithms](https://mlsanity.com/paper/2106.11344/) (2021)
- [Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap](https://mlsanity.com/paper/1711.11286/) (2017)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [Correcting Sociodemographic Selection Biases for Population Prediction from Social Media](https://mlsanity.com/paper/1911.03855/) (2019)
- [Semiparametric Estimation with Data Missing Not at Random Using an Instrumental Variable](https://mlsanity.com/paper/1607.03197/) (2016)
- [Fast and Accurate Importance Weighting for Correcting Sample Bias](https://mlsanity.com/paper/2209.04215/) (2022)
- [A Python Library For Empirical Calibration](https://mlsanity.com/paper/1906.11920/) (2019)
- [A Framework for Understanding Selection Bias in Real-World Healthcare Data](https://mlsanity.com/paper/2304.04652/) (2023)
- [A tutorial comparing different covariate balancing methods with an application evaluating the causal effects of substance use treatment programs for adolescents](https://mlsanity.com/paper/2010.09563/) (2020)

## Abstract (arXiv)

Selection bias is a common and often unavoidable aspect of real-world data that challenges the generalizability of machine learning models. When models trained on biased data are deployed in the broader target population, poor model generalization may lead to real harm, particularly in high-risk settings such as healthcare. This risk highlights the need for practitioners to reliably assess model generalizability prior to deployment. However, existing methods for predicting model performance rely on unrealistic access to the target distribution or knowledge of the selection mechanism causing bias. To address these limitations, we propose a novel upper bound on the worst-case model performance on the target population under the realistic setting where the selection mechanism and the target population data are only partially observed. We demonstrate the validity and practical utility of our method through experiments on fully synthetic data, semi-synthetic data derived from the All of Us Research Program, and real-world selection bias in MIMIC-IV. Our work offers a principled and practical tool to estimate the impact of selection bias in an otherwise intractable setting, thereby enabling practitioners to build safer and more generalizable models in healthcare and beyond.