# Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making

arXiv: https://arxiv.org/abs/2301.05809 · HTML: https://mlsanity.com/paper/2301.05809/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 111 references that are in the map.
- First posted on arXiv 14 Jan 2023.

## Builds on (most cited first, 28 of 28)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?](https://mlsanity.com/paper/2005.01831/) (2020)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems](https://mlsanity.com/paper/2001.08298/) (2020)
- [Learning to Complement Humans](https://mlsanity.com/paper/2005.00582/) (2020)
- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- [To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making](https://mlsanity.com/paper/2102.09692/) (2021)
- ['It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions](https://mlsanity.com/paper/1801.10408/) (2018)
- ["Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans](https://mlsanity.com/paper/2001.05871/) (2020)
- [Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation](https://mlsanity.com/paper/2204.11788/) (2022)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Deciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making](https://mlsanity.com/paper/2010.07938/) (2020)
- [Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning](https://mlsanity.com/paper/2202.05402/) (2022)
- [Do Humans Trust Advice More if it Comes from AI? An Analysis of Human-AI Interactions](https://mlsanity.com/paper/2107.07015/) (2021)
- [Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging](https://mlsanity.com/paper/2205.09696/) (2022)
- [Teaching Humans When To Defer to a Classifier via Exemplars](https://mlsanity.com/paper/2111.11297/) (2021)
- [Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work](https://mlsanity.com/paper/2203.11085/) (2022)
- [On Cognitive Preferences and the Plausibility of Rule-based Models](https://mlsanity.com/paper/1803.01316/) (2018)
- [Expert-Augmented Machine Learning](https://mlsanity.com/paper/1903.09731/) (2019)
- [Insurance Premium Prediction via Gradient Tree-Boosted Tweedie Compound Poisson Models](https://mlsanity.com/paper/1508.06378/) (2015)
- [Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork](https://mlsanity.com/paper/2004.13102/) (2020)

## Built on by (most cited first, 30 of 67)

- [Determinants of LLM-assisted Decision-Making](https://mlsanity.com/paper/2402.17385/) (2024)
- [Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant](https://mlsanity.com/paper/2502.01390/) (2025)
- [Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making](https://mlsanity.com/paper/2403.16812/) (2024)
- [Complementarity in Human-AI Collaboration: Concept, Sources, and Evidence](https://mlsanity.com/paper/2404.00029/) (2024)
- [A Decision Theoretic Framework for Measuring AI Reliance](https://mlsanity.com/paper/2401.15356/) (2024)
- [A Diachronic Perspective on User Trust in AI under Uncertainty](https://mlsanity.com/paper/2310.13544/) (2023)
- [A Systematic Review on Fostering Appropriate Trust in Human-AI Interaction](https://mlsanity.com/paper/2311.06305/) (2023)
- [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023)
- [As Confidence Aligns: Exploring the Effect of AI Confidence on Human Self-confidence in Human-AI Decision Making](https://mlsanity.com/paper/2501.12868/) (2025)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Intelligent AI Delegation](https://mlsanity.com/paper/2602.11865/) (2026)
- [Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure](https://mlsanity.com/paper/2306.07458/) (2023)
- [Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making](https://mlsanity.com/paper/2401.07058/) (2024)
- [Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning](https://mlsanity.com/paper/2403.05911/) (2024)
- [Modulating Language Model Experiences through Frictions](https://mlsanity.com/paper/2407.12804/) (2024)
- [AI Chatbots for Mental Health: Values and Harms from Lived Experiences of Depression](https://mlsanity.com/paper/2504.18932/) (2025)
- [Mapping Caregiver Needs to AI Chatbot Design: Strengths and Gaps in Mental Health Support for Alzheimer's and Dementia Caregivers](https://mlsanity.com/paper/2506.15047/) (2025)
- [Measuring and mitigating overreliance to build human-compatible AI](https://mlsanity.com/paper/2509.08010/) (2025)
- [How should AI decisions be explained? Requirements for Explanations from the Perspective of European Law](https://mlsanity.com/paper/2404.12762/) (2024)
- [DiscipLink: Unfolding Interdisciplinary Information Seeking Process via Human-AI Co-Exploration](https://mlsanity.com/paper/2408.00447/) (2024)
- [Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary](https://mlsanity.com/paper/2411.10461/) (2024)
- [Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI](https://mlsanity.com/paper/2308.07213/) (2023)
- [Decoding AI's Nudge: A Unified Framework to Predict Human Behavior in AI-assisted Decision Making](https://mlsanity.com/paper/2401.05840/) (2024)
- [From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis](https://mlsanity.com/paper/2502.11919/) (2025)
- [Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students](https://mlsanity.com/paper/2401.14915/) (2024)
- [Understanding the Effects of Miscalibrated AI Confidence on User Trust, Reliance, and Decision Efficacy](https://mlsanity.com/paper/2402.07632/) (2024)
- [Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance](https://mlsanity.com/paper/2502.13321/) (2025)
- [May I Ask a Follow-up Question? Understanding the Benefits of Conversations in Neural Network Explainability](https://mlsanity.com/paper/2309.13965/) (2023)
- [Beyond Recommender: An Exploratory Study of the Effects of Different AI Roles in AI-Assisted Decision Making](https://mlsanity.com/paper/2403.01791/) (2024)
- [Epistemology gives a Future to Complementarity in Human-AI Interactions](https://mlsanity.com/paper/2601.09871/) (2026)

## Abstract (arXiv)

In AI-assisted decision-making, it is critical for human decision-makers to know when to trust AI and when to trust themselves. However, prior studies calibrated human trust only based on AI confidence indicating AI's correctness likelihood (CL) but ignored humans' CL, hindering optimal team decision-making. To mitigate this gap, we proposed to promote humans' appropriate trust based on the CL of both sides at a task-instance level. We first modeled humans' CL by approximating their decision-making models and computing their potential performance in similar instances. We demonstrated the feasibility and effectiveness of our model via two preliminary studies. Then, we proposed three CL exploitation strategies to calibrate users' trust explicitly/implicitly in the AI-assisted decision-making process. Results from a between-subjects experiment (N=293) showed that our CL exploitation strategies promoted more appropriate human trust in AI, compared with only using AI confidence. We further provided practical implications for more human-compatible AI-assisted decision-making.