# Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness

arXiv: https://arxiv.org/abs/2202.08821 · HTML: https://mlsanity.com/paper/2202.08821/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 21 of 21)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [The Algorithmic Automation Problem: Prediction, Triage, and Human Effort](https://mlsanity.com/paper/1903.12220/) (2019)
- [Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency](https://mlsanity.com/paper/2006.16736/) (2020)
- [Differentiable Learning Under Triage](https://mlsanity.com/paper/2103.08902/) (2021)
- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [Towards Unbiased and Accurate Deferral to Multiple Experts](https://mlsanity.com/paper/2102.13004/) (2021)
- [Fairness Under Composition](https://mlsanity.com/paper/1806.06122/) (2018)
- [Classification Under Human Assistance](https://mlsanity.com/paper/2006.11845/) (2020)
- [Combining Human Predictions with Model Probabilities via Confusion Matrices and Calibration](https://mlsanity.com/paper/2109.14591/) (2021)
- [An Empirical Analysis of Backward Compatibility in Machine Learning Systems](https://mlsanity.com/paper/2008.04572/) (2020)
- [Do Humans Trust Advice More if it Comes from AI? An Analysis of Human-AI Interactions](https://mlsanity.com/paper/2107.07015/) (2021)
- [Individual Fairness in Pipelines](https://mlsanity.com/paper/2004.05167/) (2020)
- [Investigating Human + Machine Complementarity for Recidivism Predictions](https://mlsanity.com/paper/1808.09123/) (2018)
- [Enhancing the Accuracy and Fairness of Human Decision Making](https://mlsanity.com/paper/1805.10318/) (2018)
- [On the Fairness of Machine-Assisted Human Decisions](https://mlsanity.com/paper/2110.15310/) (2021)
- [Designing Closed Human-in-the-loop Deferral Pipelines](https://mlsanity.com/paper/2202.04718/) (2022)

## Built on by (most cited first, 30 of 45)

- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [Complementarity in Human-AI Collaboration: Concept, Sources, and Evidence](https://mlsanity.com/paper/2404.00029/) (2024)
- [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023)
- [A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementarity](https://mlsanity.com/paper/2204.10806/) (2022)
- [From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered](https://mlsanity.com/paper/2506.07461/) (2025)
- [Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)](https://mlsanity.com/paper/2407.12858/) (2024)
- [Human Expertise in Algorithmic Prediction](https://mlsanity.com/paper/2402.00793/) (2024)
- [Algorithmic Assistance with Recommendation-Dependent Preferences](https://mlsanity.com/paper/2208.07626/) (2022)
- [CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding](https://mlsanity.com/paper/2512.04111/) (2025)
- [Why is plausibility surprisingly problematic as an XAI criterion?](https://mlsanity.com/paper/2303.17707/) (2023)
- [Capturing Humans' Mental Models of AI: An Item Response Theory Approach](https://mlsanity.com/paper/2305.09064/) (2023)
- [A No Free Lunch Theorem for Human-AI Collaboration](https://mlsanity.com/paper/2411.15230/) (2024)
- [AI Agents for Inventory Control: Human-LLM-OR Complementarity](https://mlsanity.com/paper/2602.12631/) (2026)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023)
- [Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge](https://mlsanity.com/paper/2308.15700/) (2023)
- [A Unifying Post-Processing Framework for Multi-Objective Learn-to-Defer Problems](https://mlsanity.com/paper/2407.12710/) (2024)
- [Epistemology gives a Future to Complementarity in Human-AI Interactions](https://mlsanity.com/paper/2601.09871/) (2026)
- [Algorithmic Decision-Making Safeguarded by Human Knowledge](https://mlsanity.com/paper/2211.11028/) (2022)
- [When Are Two Lists Better than One?: Benefits and Harms in Joint Decision-making](https://mlsanity.com/paper/2308.11721/) (2023)
- [Learning To Guide Human Decision Makers With Vision-Language Models](https://mlsanity.com/paper/2403.16501/) (2024)
- [Designing Algorithmic Recommendations to Achieve Human-AI Complementarity](https://mlsanity.com/paper/2405.01484/) (2024)
- [How to Enable Effective Cooperation Between Humans and NLP Models: A Survey of Principles, Formalizations, and Beyond](https://mlsanity.com/paper/2501.05714/) (2025)
- [Human-AI Governance (HAIG): A Trust-Utility Approach](https://mlsanity.com/paper/2505.01651/) (2025)
- [Expanding Knowledge Graphs with Humans in the Loop](https://mlsanity.com/paper/2212.05189/) (2022)
- [Persuasion, Delegation, and Private Information in Algorithm-Assisted Decisions](https://mlsanity.com/paper/2402.09384/) (2024)
- [Collaborative Intelligence in Sequential Experiments: A Human-in-the-Loop Framework for Drug Discovery](https://mlsanity.com/paper/2405.03942/) (2024)
- [Integrating Expert Judgment and Algorithmic Decision Making: An Indistinguishability Framework](https://mlsanity.com/paper/2410.08783/) (2024)
- [Decision-aid or Controller? Steering Human Decision Makers with Algorithms](https://mlsanity.com/paper/2303.13712/) (2023)
- ["It depends": Configuring AI to Improve Clinical Usefulness Across Contexts](https://mlsanity.com/paper/2407.11978/) (2024)

## Abstract (arXiv)

Much of machine learning research focuses on predictive accuracy: given a task, create a machine learning model (or algorithm) that maximizes accuracy. In many settings, however, the final prediction or decision of a system is under the control of a human, who uses an algorithm's output along with their own personal expertise in order to produce a combined prediction. One ultimate goal of such collaborative systems is "complementarity": that is, to produce lower loss (equivalently, greater payoff or utility) than either the human or algorithm alone. However, experimental results have shown that even in carefully-designed systems, complementary performance can be elusive. Our work provides three key contributions. First, we provide a theoretical framework for modeling simple human-algorithm systems and demonstrate that multiple prior analyses can be expressed within it. Next, we use this model to prove conditions where complementarity is impossible, and give constructive examples of where complementarity is achievable. Finally, we discuss the implications of our findings, especially with respect to the fairness of a classifier. In sum, these results deepen our understanding of key factors influencing the combined performance of human-algorithm systems, giving insight into how algorithmic tools can best be designed for collaborative environments.