# Learning with Selectively Labeled Data from Multiple Decision-makers

arXiv: https://arxiv.org/abs/2306.07566 · HTML: https://mlsanity.com/paper/2306.07566/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 40 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 8 of 8)

- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Counterfactual Probabilities: Computational Methods, Bounds and Applications](https://mlsanity.com/paper/1302.6784/) (2013)
- [Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables](https://mlsanity.com/paper/1611.09925/) (2016)
- [A semiparametric instrumental variable approach to optimal treatment regimes under endogeneity](https://mlsanity.com/paper/1911.09260/) (2019)
- [Learning under selective labels in the presence of expert consistency](https://mlsanity.com/paper/1807.00905/) (2018)
- [Estimating Optimal Treatment Rules with an Instrumental Variable: A Partial Identification Learning Approach](https://mlsanity.com/paper/2002.02579/) (2020)
- [Leveraging Expert Consistency to Improve Algorithmic Decision Support](https://mlsanity.com/paper/2101.09648/) (2021)

## Built on by (most cited first, 3 of 3)

- [Predictive Performance Comparison of Decision Policies Under Confounding](https://mlsanity.com/paper/2404.00848/) (2024)
- [Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias](https://mlsanity.com/paper/2606.14506/) (2026)
- [Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector](https://mlsanity.com/paper/2310.19091/) (2023)

## Abstract (arXiv)

We study the problem of classification with selectively labeled data, whose distribution may differ from the full population due to historical decision-making. We exploit the fact that in many applications historical decisions were made by multiple decision-makers, each with different decision rules. We analyze this setup under a principled instrumental variable (IV) framework and rigorously study the identification of classification risk. We establish conditions for the exact identification of classification risk and derive tight partial identification bounds when exact identification fails. We further propose a unified cost-sensitive learning (UCL) approach to learn classifiers robust to selection bias in both identification settings. Finally, we theoretically and numerically validate the efficacy of our proposed method.