# Multi-Target Multiplicity: Flexibility and Fairness in Target Specification under Resource Constraints

arXiv: https://arxiv.org/abs/2306.13738 · HTML: https://mlsanity.com/paper/2306.13738/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 23 Jun 2023.

## Builds on (most cited first, 22 of 22)

- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [On Counterfactual Explanations under Predictive Multiplicity](https://mlsanity.com/paper/2006.13132/) (2020)
- [Characterizing Fairness Over the Set of Good Models Under Selective Labels](https://mlsanity.com/paper/2101.00352/) (2021)
- [Rashomon Capacity: A Metric for Predictive Multiplicity in Classification](https://mlsanity.com/paper/2206.01295/) (2022)
- [Predictive Multiplicity in Probabilistic Classification](https://mlsanity.com/paper/2206.01131/) (2022)
- [Leave-one-out Unfairness](https://mlsanity.com/paper/2107.10171/) (2021)
- [Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning](https://mlsanity.com/paper/2106.02705/) (2021)
- [Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics](https://mlsanity.com/paper/2005.07572/) (2020)
- [A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms](https://mlsanity.com/paper/2206.14983/) (2022)
- [The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimization](https://mlsanity.com/paper/2202.11776/) (2022)
- [From Optimizing Engagement to Measuring Value](https://mlsanity.com/paper/2008.12623/) (2020)
- [Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Models](https://mlsanity.com/paper/1901.03209/) (2019)
- [Loss-Aversively Fair Classification](https://mlsanity.com/paper/2105.04273/) (2021)
- [Algorithmic Fairness in Business Analytics: Directions for Research and Practice](https://mlsanity.com/paper/2207.10991/) (2022)
- [Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare](https://mlsanity.com/paper/2202.01906/) (2022)
- [Reconciling Individual Probability Forecasts](https://mlsanity.com/paper/2209.01687/) (2022)
- [Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models](https://mlsanity.com/paper/2206.09875/) (2022)

## Built on by (most cited first, 11 of 11)

- [Predictive Churn with the Set of Good Models](https://mlsanity.com/paper/2402.07745/) (2024)
- [Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Time](https://mlsanity.com/paper/2503.01087/) (2025)
- [One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisions](https://mlsanity.com/paper/2308.16681/) (2023)
- [An Experimental Study on the Rashomon Effect of Balancing Methods in Imbalanced Classification](https://mlsanity.com/paper/2405.01557/) (2024)
- [What Constitutes a Less Discriminatory Algorithm?](https://mlsanity.com/paper/2412.18138/) (2024)
- [Observational Multiplicity](https://mlsanity.com/paper/2507.23136/) (2025)
- [Predictive Multiplicity in Survival Models: A Method for Quantifying Model Uncertainty in Predictive Maintenance Applications](https://mlsanity.com/paper/2504.12156/) (2025)
- [Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Tasks](https://mlsanity.com/paper/2507.02819/) (2025)
- [Reconciling Predictive Multiplicity in Practice](https://mlsanity.com/paper/2501.16549/) (2025)
- [Data as a Lever: A Neighbouring Datasets Perspective on Predictive Multiplicity](https://mlsanity.com/paper/2510.21303/) (2025)
- [Falsifying Discriminant Validity of Predictive Algorithms](https://mlsanity.com/paper/2601.17146/) (2026)

## Abstract (arXiv)

Prediction models have been widely adopted as the basis for decision-making in domains as diverse as employment, education, lending, and health. Yet, few real world problems readily present themselves as precisely formulated prediction tasks. In particular, there are often many reasonable target variable options. Prior work has argued that this is an important and sometimes underappreciated choice, and has also shown that target choice can have a significant impact on the fairness of the resulting model. However, the existing literature does not offer a formal framework for characterizing the extent to which target choice matters in a particular task. Our work fills this gap by drawing connections between the problem of target choice and recent work on predictive multiplicity. Specifically, we introduce a conceptual and computational framework for assessing how the choice of target affects individuals' outcomes and selection rate disparities across groups. We call this multi-target multiplicity. Along the way, we refine the study of single-target multiplicity by introducing notions of multiplicity that respect resource constraints -- a feature of many real-world tasks that is not captured by existing notions of predictive multiplicity. We apply our methods on a healthcare dataset, and show that the level of multiplicity that stems from target variable choice can be greater than that stemming from nearly-optimal models of a single target.