# Selective Prediction Reduces the Negative Effects of Automation Bias Overall but Increases False Negatives

arXiv: https://arxiv.org/abs/2508.07617 · HTML: https://mlsanity.com/paper/2508.07617/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 11 Aug 2025.

## Builds on (most cited first, 11 of 11)

- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Efficient Pipeline for Camera Trap Image Review](https://mlsanity.com/paper/1907.06772/) (2019)
- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [Who Should Predict? Exact Algorithms For Learning to Defer to Humans](https://mlsanity.com/paper/2301.06197/) (2023)
- [Role of Human-AI Interaction in Selective Prediction](https://mlsanity.com/paper/2112.06751/) (2021)
- [Selective Regression Under Fairness Criteria](https://mlsanity.com/paper/2110.15403/) (2021)
- [Trustworthy and Practical AI for Healthcare: A Guided Deferral System with Large Language Models](https://mlsanity.com/paper/2406.07212/) (2024)
- [Combining chest X-rays and electronic health record (EHR) data using machine learning to diagnose acute respiratory failure](https://mlsanity.com/paper/2108.12530/) (2021)
- [ASPEST: Bridging the Gap Between Active Learning and Selective Prediction](https://mlsanity.com/paper/2304.03870/) (2023)
- [Joint Training for Selective Prediction](https://mlsanity.com/paper/2410.24029/) (2024)

## Built on by (most cited first, 2 of 2)

- [Source-Dependent Deference in Medical Imaging Agents Under Falsified Findings: A Pilot Audit](https://mlsanity.com/paper/2608.29800/) (2026)
- [OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One](https://mlsanity.com/paper/2610.00381/) (2026)

## Abstract (arXiv)

AI has the potential to augment human decision making. However, even high-performing models can produce inaccurate predictions when deployed. These inaccuracies, combined with automation bias, where humans overrely on AI predictions, can result in worse decisions. Selective prediction, in which potentially unreliable model predictions are hidden from users, has been proposed as a solution. This approach assumes that when AI abstains and informs the user so, humans make decisions as they would without AI involvement. To test this assumption, we study the effects of selective prediction on human decisions in a clinical context. We conducted a user study of 259 clinicians tasked with diagnosing and treating hospitalized patients. We compared their baseline performance without any AI involvement to their AI-assisted accuracy with and without selective prediction. Our findings indicate that selective prediction mitigates the negative effects of inaccurate AI in terms of decision accuracy. Compared to no AI assistance, clinician accuracy declined when shown inaccurate AI predictions (66% [95% CI: 56%-75%] vs. 56% [95% CI: 46%-66%]), but recovered under selective prediction (64% [95% CI: 54%-73%]). However, while selective prediction nearly maintains overall accuracy, our results suggest that it alters patterns of mistakes: when informed the AI abstains, clinicians underdiagnose (18% increase in missed diagnoses) and undertreat (35% increase in missed treatments) compared to no AI input at all. Our findings underscore the importance of empirically validating assumptions about how humans engage with AI within human-AI systems.