A Systematic Review on Fostering Appropriate Trust in Human-AI Interaction
Not on the map. Why?
Abstract (arXiv)
Appropriate Trust in Artificial Intelligence (AI) systems has rapidly become an important area of focus for both researchers and practitioners. Various approaches have been used to achieve it, such as confidence scores, explanations, trustworthiness cues, or uncertainty communication. However, a comprehensive understanding of the field is lacking due to the diversity of perspectives arising from various backgrounds that influence it and the lack of a single definition for appropriate trust. To investigate this topic, this paper presents a systematic review to identify current practices in building appropriate trust, different ways to measure it, types of tasks used, and potential challenges associated with it. We also propose a Belief, Intentions, and Actions (BIA) mapping to study commonalities and differences in the concepts related to appropriate trust by (a) describing the existing disagreements on defining appropriate trust, and (b) providing an overview of the concepts and definitions related to appropriate trust in AI from the existing literature. Finally, the challenges identified in studying appropriate trust are discussed, and observations are summarized as current trends, potential gaps, and research opportunities for future work. Overall, the paper provides insights into the complex concept of appropriate trust in human-AI interaction and presents research opportunities to advance our understanding on this topic.
Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 25 of its 218 references that are in the map. First posted on arXiv 8 Nov 2023.
Builds on
25 of 218 references.
References · 25
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
- 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
- 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
- 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
- 2018Manipulating and Measuring Model Interpretabilitycited by 256
- 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 166
- 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
- 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
- 2019"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanationscited by 111
- 2020"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humanscited by 86
- 2023Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanationscited by 80
13 more
- 2019The relationship between trust in AI and trustworthy machine learning technologiescited by 78
- 2023Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Makingcited by 67
- 2023Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision supportcited by 65
- 2021Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Makingcited by 58
- 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
- 2018Establishing Appropriate Trust via Critical Statescited by 47
- 2023Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Carecited by 37
- 2020Modeling and Predicting Trust Dynamics in Human-Robot Teaming: A Bayesian Inference Approachcited by 14
- 2022Are we measuring trust correctly in explainability, interpretability, and transparency research?cited by 11
- 2020Toward Adaptive Trust Calibration for Level 2 Driving Automationcited by 4
- 2021An Investigation of Drivers' Dynamic Situational Trust in Conditionally Automated Drivingcited by 4
- 2023A Mental Model Based Framework of Trustcited by 3
- 2023Human-centered trust framework: An HCI perspectivecited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 14; 2026: 6.
21 papers.
Sort:
By area:
- 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
- 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 49
- 2025Agentic AI Software Engineers: Programming with Trustcited by 19
- 2025AI, Help Me Thinkunicodex2014but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Supportcited by 12
- 2025Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articlescited by 6
- 2025Human-Centered AI Communication in Co-Creativity: An Initial Framework and Insightscited by 3
- 2025Improving User Experience with FAICO: Towards a Framework for AI Communication in Human-AI Co-Creativitycited by 2
- 2025Classifying Epistemic Relationships in Human-AI Interaction: An Exploratory Approachcited by 2
- 2024Contextualized AI for Cyber Defense: An Automated Survey using LLMscited by 1
- 2026What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safetycited by 1
- 2026Empowering 9-1-1 Calltaking Training with Generative AI: Experiences and Lessons Learnedcited by 1
- 2026A Methodology for Auditable Trustworthiness Levels in AI Lifecycle Governancecited by 1
9 more
- 2025Once Upon an AI: Six Scaffolds for Child-AI Interaction Design, Inspired by Disneycited by 0
- 2025"Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Studycited by 0
- 2025Barrier Function Overrides For Non-Convex Fixed Wing Flight Control and Self-Driving Carscited by 0
- 2025Conformal Prediction and Trustworthy AIcited by 0
- 2025In Generative AI We (Dis)Trust? Computational Analysis of Trust and Distrust in Reddit Discussionscited by 0
- 2025Does In-IDE Calibration of Large Language Models work at Scale?cited by 0
- 2026When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoptioncited by 0
- 2026"I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generationcited by 0
- 2026Engagement Intensity as a Learner-Modeling Signal for Adaptive AI Ethics Instructioncited by 0
- 2026A Methodology for Auditable Trustworthiness Levels in AI Lifecycle Governance17 Jul 2026
- 2026Engagement Intensity as a Learner-Modeling Signal for Adaptive AI Ethics Instruction16 Jun 2026
- 2026"I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generation1 May 2026
- 2026Empowering 9-1-1 Calltaking Training with Generative AI: Experiences and Lessons Learned30 Jan 2026
- 2026When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoption29 Jan 2026
- 2026What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safety26 Jan 2026
- 2025Does In-IDE Calibration of Large Language Models work at Scale?26 Oct 2025
- 2025Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articles24 Oct 2025
- 2025In Generative AI We (Dis)Trust? Computational Analysis of Trust and Distrust in Reddit Discussions17 Oct 2025
- 2025Conformal Prediction and Trustworthy AI9 Aug 2025
- 2025Classifying Epistemic Relationships in Human-AI Interaction: An Exploratory Approach2 Aug 2025
- 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents23 Jun 2025
9 more
- 2025Human-Centered AI Communication in Co-Creativity: An Initial Framework and Insights23 May 2025
- 2025Barrier Function Overrides For Non-Convex Fixed Wing Flight Control and Self-Driving Cars8 May 2025
- 2025"Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Study15 Apr 2025
- 2025Once Upon an AI: Six Scaffolds for Child-AI Interaction Design, Inspired by Disney11 Apr 2025
- 2025AI, Help Me Thinkunicodex2014but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Support9 Apr 2025
- 2025Improving User Experience with FAICO: Towards a Framework for AI Communication in Human-AI Co-Creativity3 Apr 2025
- 2025Agentic AI Software Engineers: Programming with Trust19 Feb 2025
- 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant3 Feb 2025
- 2024Contextualized AI for Cyber Defense: An Automated Survey using LLMs20 Sep 2024
- 2025Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articlescited by 6
- 2025Human-Centered AI Communication in Co-Creativity: An Initial Framework and Insightscited by 3
- 2025Improving User Experience with FAICO: Towards a Framework for AI Communication in Human-AI Co-Creativitycited by 2
- 2025Classifying Epistemic Relationships in Human-AI Interaction: An Exploratory Approachcited by 2
- 2024Contextualized AI for Cyber Defense: An Automated Survey using LLMscited by 1
- 2026What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safetycited by 1
- 2026Empowering 9-1-1 Calltaking Training with Generative AI: Experiences and Lessons Learnedcited by 1
- 2026A Methodology for Auditable Trustworthiness Levels in AI Lifecycle Governancecited by 1
- 2025Once Upon an AI: Six Scaffolds for Child-AI Interaction Design, Inspired by Disneycited by 0
- 2025"Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Studycited by 0
- 2025Conformal Prediction and Trustworthy AIcited by 0
- 2025In Generative AI We (Dis)Trust? Computational Analysis of Trust and Distrust in Reddit Discussionscited by 0
12 most cited of 15 All 15 (JSON)
- 2026A Methodology for Auditable Trustworthiness Levels in AI Lifecycle Governance17 Jul 2026
- 2026Engagement Intensity as a Learner-Modeling Signal for Adaptive AI Ethics Instruction16 Jun 2026
- 2026Empowering 9-1-1 Calltaking Training with Generative AI: Experiences and Lessons Learned30 Jan 2026
- 2026When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoption29 Jan 2026
- 2026What is Safety? Corporate Discourse, Power, and the Politics of Generative AI Safety26 Jan 2026
- 2025Does In-IDE Calibration of Large Language Models work at Scale?26 Oct 2025
- 2025Understanding AI Trustworthiness: A Scoping Review of AIES & FAccT Articles24 Oct 2025
- 2025In Generative AI We (Dis)Trust? Computational Analysis of Trust and Distrust in Reddit Discussions17 Oct 2025
- 2025Conformal Prediction and Trustworthy AI9 Aug 2025
- 2025Classifying Epistemic Relationships in Human-AI Interaction: An Exploratory Approach2 Aug 2025
- 2025Human-Centered AI Communication in Co-Creativity: An Initial Framework and Insights23 May 2025
- 2025"Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Study15 Apr 2025
12 newest of 15 All 15 (JSON)