# A Unifying Post-Processing Framework for Multi-Objective Learn-to-Defer Problems

arXiv: https://arxiv.org/abs/2407.12710 · HTML: https://mlsanity.com/paper/2407.12710/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 70 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 30 of 32)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [The Algorithmic Automation Problem: Prediction, Triage, and Human Effort](https://mlsanity.com/paper/1903.12220/) (2019)
- [Differentiable Learning Under Triage](https://mlsanity.com/paper/2103.08902/) (2021)
- [Who Should Predict? Exact Algorithms For Learning to Defer to Humans](https://mlsanity.com/paper/2301.06197/) (2023)
- [Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification](https://mlsanity.com/paper/1906.05082/) (2019)
- [Classification with Rejection Based on Cost-sensitive Classification](https://mlsanity.com/paper/2010.11748/) (2020)
- [Fair and Optimal Classification via Post-Processing](https://mlsanity.com/paper/2211.01528/) (2022)
- [Sample Efficient Learning of Predictors that Complement Humans](https://mlsanity.com/paper/2207.09584/) (2022)
- [Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness](https://mlsanity.com/paper/2202.08821/) (2022)
- [Fairness Under Composition](https://mlsanity.com/paper/1806.06122/) (2018)
- [Classification Under Human Assistance](https://mlsanity.com/paper/2006.11845/) (2020)
- [Regression Under Human Assistance](https://mlsanity.com/paper/1909.02963/) (2019)
- [Selective Classification via One-Sided Prediction](https://mlsanity.com/paper/2010.07853/) (2020)
- [Selective Classification Can Magnify Disparities Across Groups](https://mlsanity.com/paper/2010.14134/) (2020)
- [Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithms](https://mlsanity.com/paper/2310.14772/) (2023)
- [Learning to Defer to a Population: A Meta-Learning Approach](https://mlsanity.com/paper/2403.02683/) (2024)
- [In Defense of Softmax Parametrization for Calibrated and Consistent Learning to Defer](https://mlsanity.com/paper/2311.01106/) (2023)
- [Bayes-Optimal Classifiers under Group Fairness](https://mlsanity.com/paper/2202.09724/) (2022)
- [Investigating Human + Machine Complementarity for Recidivism Predictions](https://mlsanity.com/paper/1808.09123/) (2018)
- [Unprocessing Seven Years of Algorithmic Fairness](https://mlsanity.com/paper/2306.07261/) (2023)
- [Plugin estimators for selective classification with out-of-distribution detection](https://mlsanity.com/paper/2301.12386/) (2023)
- [Post-hoc Bias Scoring Is Optimal For Fair Classification](https://mlsanity.com/paper/2310.05725/) (2023)
- [Consistent Multiclass Algorithms for Complex Metrics and Constraints](https://mlsanity.com/paper/2210.09695/) (2022)
- [Regression with Cost-based Rejection](https://mlsanity.com/paper/2311.04550/) (2023)
- [Bayes-Optimal Fair Classification with Linear Disparity Constraints via Pre-, In-, and Post-processing](https://mlsanity.com/paper/2402.02817/) (2024)

## Built on by (most cited first, 5 of 5)

- [Realizable H-Consistent and Bayes-Consistent Loss Functions for Learning to Defer](https://mlsanity.com/paper/2407.13732/) (2024)
- [Mastering Multiple-Expert Routing: Realizable H-Consistency and Strong Guarantees for Learning to Defer](https://mlsanity.com/paper/2506.20650/) (2025)
- [Budgeted Multiple-Expert Deferral](https://mlsanity.com/paper/2510.26706/) (2025)
- [When More Experts Hurt: Underfitting in Multi-Expert Learning to Defer](https://mlsanity.com/paper/2602.17144/) (2026)
- [Density-Ratio Losses for Post-Hoc Learning to Defer](https://mlsanity.com/paper/2605.19557/) (2026)

## Abstract (arXiv)

Learn-to-Defer is a paradigm that enables learning algorithms to work not in isolation but as a team with human experts. In this paradigm, we permit the system to defer a subset of its tasks to the expert. Although there are currently systems that follow this paradigm and are designed to optimize the accuracy of the final human-AI team, the general methodology for developing such systems under a set of constraints (e.g., algorithmic fairness, expert intervention budget, defer of anomaly, etc.) remains largely unexplored. In this paper, using a d-dimensional generalization to the fundamental lemma of Neyman and Pearson (d-GNP), we obtain the Bayes optimal solution for learn-to-defer systems under various constraints. Furthermore, we design a generalizable algorithm to estimate that solution and apply this algorithm to the COMPAS and ACSIncome datasets. Our algorithm shows improvements in terms of constraint violation over a set of baselines.