# Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative

arXiv: https://arxiv.org/abs/2103.04725 · HTML: https://mlsanity.com/paper/2103.04725/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Mar 2021.

## Built on by (most cited first, 28 of 28)

- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [Improving Expert Predictions with Conformal Prediction](https://mlsanity.com/paper/2201.12006/) (2022)
- [Teaching Humans When To Defer to a Classifier via Exemplars](https://mlsanity.com/paper/2111.11297/) (2021)
- [Selective Explanations: Leveraging Human Input to Align Explainable AI](https://mlsanity.com/paper/2301.09656/) (2023)
- [The Impact of Imperfect XAI on Human-AI Decision-Making](https://mlsanity.com/paper/2307.13566/) (2023)
- [Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure](https://mlsanity.com/paper/2306.07458/) (2023)
- [Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling](https://mlsanity.com/paper/2401.08876/) (2024)
- [Cost-Sensitive Learning to Defer to Multiple Experts with Workload Constraints](https://mlsanity.com/paper/2403.06906/) (2024)
- [Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making](https://mlsanity.com/paper/2302.08807/) (2023)
- [Is explainable AI a race against model complexity?](https://mlsanity.com/paper/2205.10119/) (2022)
- [Decoding AI's Nudge: A Unified Framework to Predict Human Behavior in AI-assisted Decision Making](https://mlsanity.com/paper/2401.05840/) (2024)
- [A Test for Evaluating Performance in Human-Computer Systems](https://mlsanity.com/paper/2206.12390/) (2022)
- [Multi-model Ensemble Conformal Prediction in Dynamic Environments](https://mlsanity.com/paper/2411.03678/) (2024)
- [When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Tasks](https://mlsanity.com/paper/2510.05307/) (2025)
- [FiFAR: A Fraud Detection Dataset for Learning to Defer](https://mlsanity.com/paper/2312.13218/) (2023)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [ObSynth: An Interactive Synthesis System for Generating Object Models from Natural Language Specifications](https://mlsanity.com/paper/2210.11468/) (2022)
- [Ontology in Hybrid Intelligence: a concise literature review](https://mlsanity.com/paper/2303.17262/) (2023)
- [Exploring the Role of Explainability in AI-Assisted Embryo Selection](https://mlsanity.com/paper/2308.02534/) (2023)
- [What happens when reviewers receive AI feedback in their reviews?](https://mlsanity.com/paper/2602.13817/) (2026)
- [Can AI Mitigate Human Perceptual Biases? A Pilot Study](https://mlsanity.com/paper/2311.00706/) (2023)
- [Graphical Perception of Saliency-based Model Explanations](https://mlsanity.com/paper/2406.07702/) (2024)
- [Evaluation and Incident Prevention in an Enterprise AI Assistant](https://mlsanity.com/paper/2504.13924/) (2025)
- [Survey for Categorising Explainable AI Studies Using Data Analysis Task Frameworks](https://mlsanity.com/paper/2507.10208/) (2025)
- [TCM-5CEval: Extended Deep Evaluation Benchmark for LLM's Comprehensive Clinical Research Competence in Traditional Chinese Medicine](https://mlsanity.com/paper/2511.13169/) (2025)
- [Biased Minds Meet Biased AI: How Class Imbalance Shapes Appropriate Reliance and Interacts with Human Base Rate Neglect](https://mlsanity.com/paper/2511.14591/) (2025)

## Abstract (arXiv)

Automated decision support can accelerate tedious tasks as users can focus their attention where it is needed most. However, a key concern is whether users overly trust or cede agency to automation. In this paper, we investigate the effects of introducing automation to annotating clinical texts--a multi-step, error-prone task of identifying clinical concepts (e.g., procedures) in medical notes, and mapping them to labels in a large ontology. We consider two forms of decision aid: recommending which labels to map concepts to, and pre-populating annotation suggestions. Through laboratory studies, we find that 18 clinicians generally build intuition of when to rely on automation and when to exercise their own judgement. However, when presented with fully pre-populated suggestions, these expert users exhibit less agency: accepting improper mentions, and taking less initiative in creating additional annotations. Our findings inform how systems and algorithms should be designed to mitigate the observed issues.