# On the Fairness of Machine-Assisted Human Decisions

arXiv: https://arxiv.org/abs/2110.15310 · HTML: https://mlsanity.com/paper/2110.15310/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 66 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices](https://mlsanity.com/paper/1906.09208/) (2019)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [From Predictive to Prescriptive Analytics](https://mlsanity.com/paper/1402.5481/) (2014)
- [Algorithmic Fairness](https://mlsanity.com/paper/2001.09784/) (2020)
- ["How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations](https://mlsanity.com/paper/1911.06473/) (2019)
- [The Algorithmic Automation Problem: Prediction, Triage, and Human Effort](https://mlsanity.com/paper/1903.12220/) (2019)
- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [Fairness in Risk Assessment Instruments: Post-Processing to Achieve Counterfactual Equalized Odds](https://mlsanity.com/paper/2009.02841/) (2020)
- [Experimental Evaluation of Algorithm-Assisted Human Decision-Making: Application to Pretrial Public Safety Assessment](https://mlsanity.com/paper/2012.02845/) (2020)
- [Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability](https://mlsanity.com/paper/1809.04578/) (2018)
- [From Soft Classifiers to Hard Decisions: How fair can we be?](https://mlsanity.com/paper/1810.02003/) (2018)
- [The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies](https://mlsanity.com/paper/2109.01443/) (2021)
- [Paradoxes in Fair Computer-Aided Decision Making](https://mlsanity.com/paper/1711.11066/) (2017)

## Built on by (most cited first, 11 of 11)

- [Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness](https://mlsanity.com/paper/2202.08821/) (2022)
- [Algorithmic Assistance with Recommendation-Dependent Preferences](https://mlsanity.com/paper/2208.07626/) (2022)
- [EconEvals: Benchmarks and Litmus Tests for Economic Decision-Making by LLM Agents](https://mlsanity.com/paper/2503.18825/) (2025)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [Improving Stability in Decision Tree Models](https://mlsanity.com/paper/2305.17299/) (2023)
- [AI Oversight and Human Mistakes: Evidence from Centre Court](https://mlsanity.com/paper/2401.16754/) (2024)
- [Rethinking Algorithmic Fairness for Human-AI Collaboration](https://mlsanity.com/paper/2310.03647/) (2023)
- [Interventions Against Machine-Assisted Statistical Discrimination](https://mlsanity.com/paper/2310.04585/) (2023)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [AI Recommendations and Non-instrumental Image Concerns](https://mlsanity.com/paper/2504.19047/) (2025)
- [AI Model Design and Physician Liability](https://mlsanity.com/paper/2608.13618/) (2026)

## Abstract (arXiv)

When machine-learning algorithms are used in high-stakes decisions, we want to ensure that their deployment leads to fair and equitable outcomes. This concern has motivated a fast-growing literature that focuses on diagnosing and addressing disparities in machine predictions. However, many machine predictions are deployed to assist in decisions where a human decision-maker retains the ultimate decision authority. In this article, we therefore consider in a formal model and in a lab experiment how properties of machine predictions affect the resulting human decisions. In our formal model of statistical decision-making, we show that the inclusion of a biased human decision-maker can revert common relationships between the structure of the algorithm and the qualities of resulting decisions. Specifically, we document that excluding information about protected groups from the prediction may fail to reduce, and may even increase, ultimate disparities. In the lab experiment, we demonstrate how predictions informed by gender-specific information can reduce average gender disparities in decisions. While our concrete theoretical results rely on specific assumptions about the data, algorithm, and decision-maker, and the experiment focuses on a particular prediction task, our findings show more broadly that any study of critical properties of complex decision systems, such as the fairness of machine-assisted human decisions, should go beyond focusing on the underlying algorithmic predictions in isolation.