mlsanityData to 7 Oct 2026

A Diachronic Perspective on User Trust in AI under Uncertainty

arXiv 2310.13544 ↗First posted 20 Oct 2023cs.CL cs.HCcited by 23

Not on the map. Why?

Abstract (arXiv)

In a human-AI collaboration, users build a mental model of the AI system based on its reliability and how it presents its decision, e.g. its presentation of system confidence and an explanation of the output. Modern NLP systems are often uncalibrated, resulting in confidently incorrect predictions that undermine user trust. In order to build trustworthy AI, we must understand how user trust is developed and how it can be regained after potential trust-eroding events. We study the evolution of user trust in response to these trust-eroding events using a betting game. We find that even a few incorrect instances with inaccurate confidence estimates damage user trust and performance, with very slow recovery. We also show that this degradation in trust reduces the success of human-AI collaboration and that different types of miscalibration -- unconfidently correct and confidently incorrect -- have different negative effects on user trust. Our findings highlight the importance of calibration in user-facing AI applications and shed light on what aspects help users decide whether to trust the AI system.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 18 of its 55 references that are in the map. First posted on arXiv 20 Oct 2023.

Builds on

18 of 55 references.

References · 18

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  5. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  6. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  7. 2020Selective Question Answering under Domain Shiftcited by 228
  8. 2017One Model To Learn Them Allcited by 86
  9. 2023Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Makingcited by 67
  10. 2019How model accuracy and explanation fidelity influence user trustcited by 44
  11. 2019Quizbowl: The Case for Incremental Question Answeringcited by 36
  12. 2023The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answeringcited by 26
6 more
  1. 2020Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QAcited by 17
  2. 2020Modeling Trust in Human-Robot Interaction: A Surveycited by 12
  3. 2022Calibration of Machine Reading Systems at Scalecited by 11
  4. 2023Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Referencecited by 11
  5. 2021Backtranslation Feedback Improves User Confidence in MT, Not Qualitycited by 10
  6. 2020Causal Perception in Question-Answering Systemscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 6; 2026: 6.

23 papers.

Sort:

By area:

  1. 2024Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertaintycited by 106
  2. 2024Calibrating Large Language Models Using Their Generations Onlycited by 63
  3. 2025Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Modelscited by 39
  4. 2024Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluationcited by 29
  5. 2024A Survey on the Honesty of Large Language Modelscited by 21
  6. 2024AI-Assisted Human Evaluation of Machine Translationcited by 20
  7. 2026Intelligent AI Delegationcited by 16
  8. 2024Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?cited by 15
  9. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
  10. 2024Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliancecited by 15
  11. 2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14
  12. 2024The Call for Socially Aware Language Technologiescited by 12
11 more
  1. 2024Calibration Attacks: A Comprehensive Study of Adversarial Attacks on Model Confidencecited by 7
  2. 2024The Illusion of Competence: Evaluating the Effect of Explanations on Users' Mental Models of Visual Question Answering Systemscited by 6
  3. 2025Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistancecited by 6
  4. 2025Humans overrely on overconfident language models, across languagescited by 3
  5. 2025On the Robustness of Verbal Confidence of LLMs in Adversarial Attackscited by 3
  6. 2026Reference Games as a Testbed for the Alignment of Model Uncertainty and Clarification Requestscited by 2
  7. 2025PIE: Performance Interval Estimation for Free-Form Generation Taskscited by 1
  8. 2026Effects of Generative AI Errors on User Reliance Across Task Difficultycited by 1
  9. 2026Uncertainty-Aware Generation and Decision-Making Under Ambiguitycited by 1
  10. 2026Last Translation Benchmarkcited by 1
  11. 2026Align AI to Dynamic Human-AI Workflowscited by 0