# Role of Human-AI Interaction in Selective Prediction

arXiv: https://arxiv.org/abs/2112.06751 · HTML: https://mlsanity.com/paper/2112.06751/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 13 Dec 2021.

## Builds on (most cited first, 11 of 11)

- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [Learning to Complement Humans](https://mlsanity.com/paper/2005.00582/) (2020)
- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [Towards Unbiased and Accurate Deferral to Multiple Experts](https://mlsanity.com/paper/2102.13004/) (2021)
- [Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making](https://mlsanity.com/paper/2010.07938/) (2020)
- [Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes](https://mlsanity.com/paper/1904.09612/) (2019)
- [Context R-CNN: Long Term Temporal Context for Per-Camera Object Detection](https://mlsanity.com/paper/1912.03538/) (2019)
- [A deep active learning system for species identification and counting in camera trap images](https://mlsanity.com/paper/1910.09716/) (2019)

## Built on by (most cited first, 30 of 35)

- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [Human Uncertainty in Concept-Based AI Systems](https://mlsanity.com/paper/2303.12872/) (2023)
- [On the Utility of Prediction Sets in Human-AI Teams](https://mlsanity.com/paper/2205.01411/) (2022)
- [Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction](https://mlsanity.com/paper/2303.09224/) (2023)
- [The Impact of Imperfect XAI on Human-AI Decision-Making](https://mlsanity.com/paper/2307.13566/) (2023)
- [AI-Assisted Human Evaluation of Machine Translation](https://mlsanity.com/paper/2406.12419/) (2024)
- [Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic review](https://mlsanity.com/paper/2310.19778/) (2023)
- [Learning Personalized Decision Support Policies](https://mlsanity.com/paper/2304.06701/) (2023)
- [A Causal Framework for Evaluating Deferring Systems](https://mlsanity.com/paper/2405.18902/) (2024)
- [The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment](https://mlsanity.com/paper/2412.16468/) (2024)
- [A Test for Evaluating Performance in Human-Computer Systems](https://mlsanity.com/paper/2206.12390/) (2022)
- [A No Free Lunch Theorem for Human-AI Collaboration](https://mlsanity.com/paper/2411.15230/) (2024)
- [Learning Complementary Policies for Human-AI Teams](https://mlsanity.com/paper/2302.02944/) (2023)
- [Explaining and Improving Information Complementarities in Multi-Agent Decision-making](https://mlsanity.com/paper/2502.06152/) (2025)
- [Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information](https://mlsanity.com/paper/2401.04729/) (2024)
- [Human-AI Complementarity: A Goal for Amplified Oversight](https://mlsanity.com/paper/2510.26518/) (2025)
- [Selective classification using a robust meta-learning approach](https://mlsanity.com/paper/2212.05987/) (2022)
- [Towards Regulatable AI Systems: Technical Gaps and Policy Opportunities](https://mlsanity.com/paper/2306.12609/) (2023)
- [Reflections from the Workshop on AI-Assisted Decision Making for Conservation](https://mlsanity.com/paper/2307.08774/) (2023)
- [Selective Prediction Reduces the Negative Effects of Automation Bias Overall but Increases False Negatives](https://mlsanity.com/paper/2508.07617/) (2025)
- [ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making](https://mlsanity.com/paper/2602.19458/) (2026)
- [Learning to Advise Humans in High-Stakes Settings](https://mlsanity.com/paper/2210.12849/) (2022)
- [Are They the Same Picture? Adapting Concept Bottleneck Models for Human-AI Collaboration in Image Retrieval](https://mlsanity.com/paper/2407.08908/) (2024)
- [Effect of Adaptation Rate and Cost Display in a Human-AI Interaction Game](https://mlsanity.com/paper/2408.14640/) (2024)
- [Joint Training for Selective Prediction](https://mlsanity.com/paper/2410.24029/) (2024)
- [Developing A Visual-Interactive Interface for Electronic Health Record Labeling: An Explainable Machine Learning Approach](https://mlsanity.com/paper/2209.12778/) (2022)
- [Trust Your Gut: Comparing Human and Machine Inference from Noisy Visualizations](https://mlsanity.com/paper/2407.16871/) (2024)
- [How to Measure Human-AI Prediction Accuracy in Explainable AI Systems](https://mlsanity.com/paper/2409.00069/) (2024)
- [Exploiting Contextual Uncertainty of Visual Data for Efficient Training of Deep Models](https://mlsanity.com/paper/2411.01925/) (2024)
- [Toward Human-AI Complementarity Across Diverse Tasks](https://mlsanity.com/paper/2605.04070/) (2026)

## Abstract (arXiv)

Recent work has shown the potential benefit of selective prediction systems that can learn to defer to a human when the predictions of the AI are unreliable, particularly to improve the reliability of AI systems in high-stakes applications like healthcare or conservation. However, most prior work assumes that human behavior remains unchanged when they solve a prediction task as part of a human-AI team as opposed to by themselves. We show that this is not the case by performing experiments to quantify human-AI interaction in the context of selective prediction. In particular, we study the impact of communicating different types of information to humans about the AI system's decision to defer. Using real-world conservation data and a selective prediction system that improves expected accuracy over that of the human or AI system working individually, we show that this messaging has a significant impact on the accuracy of human judgements. Our results study two components of the messaging strategy: 1) Whether humans are informed about the prediction of the AI system and 2) Whether they are informed about the decision of the selective prediction system to defer. By manipulating these messaging components, we show that it is possible to significantly boost human performance by informing the human of the decision to defer, but not revealing the prediction of the AI. We therefore show that it is vital to consider how the decision to defer is communicated to a human when designing selective prediction systems, and that the composite accuracy of a human-AI team must be carefully evaluated using a human-in-the-loop framework.