# Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare

arXiv: https://arxiv.org/abs/2202.01906 · HTML: https://mlsanity.com/paper/2202.01906/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 118 references that are in the map.
- First posted on arXiv 3 Feb 2022.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [CheXclusion: Fairness gaps in deep chest X-ray classifiers](https://mlsanity.com/paper/2003.00827/) (2020)
- [Minimax Pareto Fairness: A Multi Objective Perspective](https://mlsanity.com/paper/2011.01821/) (2020)
- [Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees](https://mlsanity.com/paper/1806.06055/) (2018)
- [Fairness in Recommendation Ranking through Pairwise Comparisons](https://mlsanity.com/paper/1903.00780/) (2019)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals](https://mlsanity.com/paper/1809.04198/) (2018)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Fairness risk measures](https://mlsanity.com/paper/1901.08665/) (2019)
- [Learning Controllable Fair Representations](https://mlsanity.com/paper/1812.04218/) (2018)
- [A Scalable Discrete-Time Survival Model for Neural Networks](https://mlsanity.com/paper/1805.00917/) (2018)
- [Continuous and Discrete-Time Survival Prediction with Neural Networks](https://mlsanity.com/paper/1910.06724/) (2019)
- [Pairwise Fairness for Ranking and Regression](https://mlsanity.com/paper/1906.05330/) (2019)
- [Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints](https://mlsanity.com/paper/1807.00028/) (2018)
- [Metric Learning for Individual Fairness](https://mlsanity.com/paper/1906.00250/) (2019)
- [An Empirical Characterization of Fair Machine Learning For Clinical Risk Prediction](https://mlsanity.com/paper/2007.10306/) (2020)
- [Scalable Learning of Non-Decomposable Objectives](https://mlsanity.com/paper/1608.04802/) (2016)

## Built on by (most cited first, 16 of 16)

- [A Closer Look at AUROC and AUPRC under Class Imbalance](https://mlsanity.com/paper/2401.06091/) (2024)
- [On (assessing) the fairness of risk score models](https://mlsanity.com/paper/2302.08851/) (2023)
- [Multi-Target Multiplicity: Flexibility and Fairness in Target Specification under Resource Constraints](https://mlsanity.com/paper/2306.13738/) (2023)
- [Fill In The Gaps: Model Calibration and Generalization with Synthetic Data](https://mlsanity.com/paper/2410.10864/) (2024)
- [Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness](https://mlsanity.com/paper/2506.04193/) (2025)
- [Decision-Making under Miscalibration](https://mlsanity.com/paper/2203.09852/) (2022)
- [Fairness and Sequential Decision Making: Limits, Lessons, and Opportunities](https://mlsanity.com/paper/2301.05753/) (2023)
- [The Limits of Fair Medical Imaging AI In The Wild](https://mlsanity.com/paper/2312.10083/) (2023)
- [Decision from Suboptimal Classifiers: Excess Risk Pre- and Post-Calibration](https://mlsanity.com/paper/2503.18025/) (2025)
- [Specification Overfitting in Artificial Intelligence](https://mlsanity.com/paper/2403.08425/) (2024)
- [Critical Appraisal of Fairness Metrics in Clinical Predictive AI](https://mlsanity.com/paper/2506.17035/) (2025)
- [A Counterfactual Fair Model for Longitudinal Electronic Health Records via Deconfounder](https://mlsanity.com/paper/2308.11819/) (2023)
- [A Consequentialist Critique of Binary Classification Evaluation: Theory, Practice, and Tools](https://mlsanity.com/paper/2504.04528/) (2025)
- [Fair Machine Learning in Healthcare: A Review](https://mlsanity.com/paper/2206.14397/) (2022)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [Penalized Fair Regression for Multiple Groups in Chronic Kidney Disease](https://mlsanity.com/paper/2512.17340/) (2025)

## Abstract (arXiv)

A growing body of work uses the paradigm of algorithmic fairness to frame the development of techniques to anticipate and proactively mitigate the introduction or exacerbation of health inequities that may follow from the use of model-guided decision-making. We evaluate the interplay between measures of model performance, fairness, and the expected utility of decision-making to offer practical recommendations for the operationalization of algorithmic fairness principles for the development and evaluation of predictive models in healthcare. We conduct an empirical case-study via development of models to estimate the ten-year risk of atherosclerotic cardiovascular disease to inform statin initiation in accordance with clinical practice guidelines. We demonstrate that approaches that incorporate fairness considerations into the model training objective typically do not improve model performance or confer greater net benefit for any of the studied patient populations compared to the use of standard learning paradigms followed by threshold selection concordant with patient preferences, evidence of intervention effectiveness, and model calibration. These results hold when the measured outcomes are not subject to differential measurement error across patient populations and threshold selection is unconstrained, regardless of whether differences in model performance metrics, such as in true and false positive error rates, are present. In closing, we argue for focusing model development efforts on developing calibrated models that predict outcomes well for all patient populations while emphasizing that such efforts are complementary to transparent reporting, participatory design, and reasoning about the impact of model-informed interventions in context.