Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance
Not on the map. Why?
Abstract (arXiv)
Trust biases how users rely on AI recommendations in AI-assisted decision-making tasks, with low and high levels of trust resulting in increased under- and over-reliance, respectively. We propose that AI assistants should adapt their behavior through trust-adaptive interventions to mitigate such inappropriate reliance. For instance, when user trust is low, providing an explanation can elicit more careful consideration of the assistant's advice by the user. In two decision-making scenarios -- laypeople answering science questions and doctors making medical diagnoses -- we find that providing supporting and counter-explanations during moments of low and high trust, respectively, yields up to 38% reduction in inappropriate reliance and 20% improvement in decision accuracy. We are similarly able to reduce over-reliance by adaptively inserting forced pauses to promote deliberation. Our results highlight how AI adaptation to user trust facilitates appropriate reliance, presenting exciting avenues for improving human-AI collaboration.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 26 of its 73 references that are in the map. First posted on arXiv 18 Feb 2025.
Builds on
26 of 73 references.
References · 26
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
- 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
- 2018Manipulating and Measuring Model Interpretabilitycited by 256
- 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 166
- 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
- 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
- 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
- 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
- 2022DDXPlus: A New Dataset For Automatic Medical Diagnosiscited by 87
14 more
- 2023Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanationscited by 80
- 2023Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Makingcited by 67
- 2020Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-makingcited by 59
- 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
- 2023Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationalescited by 37
- 2023A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspectivecited by 33
- 2024"Are You Really Sure?" Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Makingcited by 26
- 2023A Diachronic Perspective on User Trust in AI under Uncertaintycited by 23
- 2023Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressurecited by 17
- 2023Learning Personalized Decision Support Policiescited by 13
- 2024On Evaluating Explanation Utility for Human-AI Decision Making in NLPcited by 9
- 2024The Illusion of Competence: Evaluating the Effect of Explanations on Users' Mental Models of Visual Question Answering Systemscited by 6
- 2024Integrating Generative AI into Financial Market Prediction for Improved Decision Makingcited by 4
- 2024The Power of Perception in Human-AI Interaction: Investigating Psychological Factors and Cognitive Biases that Shape User Belief and Behaviorcited by 1
Built on by
Citing papers per year: 2025: 4; 2026: 2.
6 papers.
- 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
- 2025Trace Length is a Simple Uncertainty Signal in Reasoning Modelscited by 16
- 2025Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translationcited by 5
- 2026Auditing and Controlling AI Agent Actions in Spreadsheetscited by 1
- 2025Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanationscited by 0
- 2026Evaluating Developmental Cognition Capabilities of LLMscited by 0
- 2026Evaluating Developmental Cognition Capabilities of LLMs8 May 2026
- 2026Auditing and Controlling AI Agent Actions in Spreadsheets22 Apr 2026
- 2025Trace Length is a Simple Uncertainty Signal in Reasoning Models12 Oct 2025
- 2025Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations30 Sep 2025
- 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered9 Jun 2025
- 2025Should I Share this Translation? Evaluating Quality Feedback for User Reliance on Machine Translation30 May 2025