# A Diachronic Perspective on User Trust in AI under Uncertainty

arXiv: https://arxiv.org/abs/2310.13544 · HTML: https://mlsanity.com/paper/2310.13544/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 18 of 18)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [Evaluating Verifiability in Generative Search Engines](https://mlsanity.com/paper/2304.09848/) (2023)
- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [One Model To Learn Them All](https://mlsanity.com/paper/1706.05137/) (2017)
- [Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making](https://mlsanity.com/paper/2301.05809/) (2023)
- [How model accuracy and explanation fidelity influence user trust](https://mlsanity.com/paper/1907.12652/) (2019)
- [Quizbowl: The Case for Incremental Question Answering](https://mlsanity.com/paper/1904.04792/) (2019)
- [The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering](https://mlsanity.com/paper/2305.16519/) (2023)
- [Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA](https://mlsanity.com/paper/2012.15075/) (2020)
- [Modeling Trust in Human-Robot Interaction: A Survey](https://mlsanity.com/paper/2011.04796/) (2020)
- [Calibration of Machine Reading Systems at Scale](https://mlsanity.com/paper/2203.10623/) (2022)
- [Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference](https://mlsanity.com/paper/2301.09008/) (2023)
- [Backtranslation Feedback Improves User Confidence in MT, Not Quality](https://mlsanity.com/paper/2104.05688/) (2021)
- [Causal Perception in Question-Answering Systems](https://mlsanity.com/paper/2012.14477/) (2020)

## Built on by (most cited first, 23 of 23)

- [Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty](https://mlsanity.com/paper/2401.06730/) (2024)
- [Calibrating Large Language Models Using Their Generations Only](https://mlsanity.com/paper/2403.05973/) (2024)
- [Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models](https://mlsanity.com/paper/2502.11028/) (2025)
- [Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation](https://mlsanity.com/paper/2410.20774/) (2024)
- [A Survey on the Honesty of Large Language Models](https://mlsanity.com/paper/2409.18786/) (2024)
- [AI-Assisted Human Evaluation of Machine Translation](https://mlsanity.com/paper/2406.12419/) (2024)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Intelligent AI Delegation](https://mlsanity.com/paper/2602.11865/) (2026)
- [Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance](https://mlsanity.com/paper/2407.07950/) (2024)
- [Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?](https://mlsanity.com/paper/2402.16102/) (2024)
- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [The Call for Socially Aware Language Technologies](https://mlsanity.com/paper/2405.02411/) (2024)
- [Calibration Attacks: A Comprehensive Study of Adversarial Attacks on Model Confidence](https://mlsanity.com/paper/2401.02718/) (2024)
- [The Illusion of Competence: Evaluating the Effect of Explanations on Users' Mental Models of Visual Question Answering Systems](https://mlsanity.com/paper/2406.19170/) (2024)
- [Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance](https://mlsanity.com/paper/2502.13321/) (2025)
- [Humans overrely on overconfident language models, across languages](https://mlsanity.com/paper/2507.06306/) (2025)
- [On the Robustness of Verbal Confidence of LLMs in Adversarial Attacks](https://mlsanity.com/paper/2507.06489/) (2025)
- [Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requests](https://mlsanity.com/paper/2601.07820/) (2026)
- [PIE: Performance Interval Estimation for Free-Form Generation Tasks](https://mlsanity.com/paper/2509.07309/) (2025)
- [Effects of Generative AI Errors on User Reliance Across Task Difficulty](https://mlsanity.com/paper/2604.04319/) (2026)
- [Uncertainty-Aware Generation and Decision-Making Under Ambiguity](https://mlsanity.com/paper/2606.30578/) (2026)
- [Align AI to Dynamic Human-AI Workflows](https://mlsanity.com/paper/2607.14240/) (2026)
- [Last Translation Benchmark](https://mlsanity.com/paper/2609.04173/) (2026)

## Abstract (arXiv)

In a human-AI collaboration, users build a mental model of the AI system based on its reliability and how it presents its decision, e.g. its presentation of system confidence and an explanation of the output. Modern NLP systems are often uncalibrated, resulting in confidently incorrect predictions that undermine user trust. In order to build trustworthy AI, we must understand how user trust is developed and how it can be regained after potential trust-eroding events. We study the evolution of user trust in response to these trust-eroding events using a betting game. We find that even a few incorrect instances with inaccurate confidence estimates damage user trust and performance, with very slow recovery. We also show that this degradation in trust reduces the success of human-AI collaboration and that different types of miscalibration -- unconfidently correct and confidently incorrect -- have different negative effects on user trust. Our findings highlight the importance of calibration in user-facing AI applications and shed light on what aspects help users decide whether to trust the AI system.