mlsanityData to 7 Oct 2026

Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI

arXiv 2010.07487 ↗First posted 15 Oct 2020cs.AI cs.CYcited by 166

Not on the map. Why?

Abstract (arXiv)

Trust is a central component of the interaction between people and AI, in that 'incorrect' levels of trust may cause misuse, abuse or disuse of the technology. But what, precisely, is the nature of trust in AI? What are the prerequisites and goals of the cognitive mechanism of trust, and how can we promote them, or assess whether they are being satisfied in a given interaction? This work aims to answer these questions. We discuss a model of trust inspired by, but not identical to, sociology's interpersonal trust (i.e., trust between people). This model rests on two key properties of the vulnerability of the user and the ability to anticipate the impact of the AI model's decisions. We incorporate a formalization of 'contractual trust', such that trust between a user and an AI is trust that some implicit or explicit contract will hold, and a formalization of 'trustworthiness' (which detaches from the notion of trustworthiness in sociology), and with it concepts of 'warranted' and 'unwarranted' trust. We then present the possible causes of warranted trust as intrinsic reasoning and extrinsic behavior, and discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted. Finally, we elucidate the connection between trust and XAI using our formalization.

Key facts. Cited by 166 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 32; 2023: 33; 2024: 30; 2025: 37; 2026: 20.

166 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2021Trustworthy AI: From Principles to Practicescited by 206
  3. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  4. 2022CRAFT: Concept Recursive Activation FacTorization for Explainabilitycited by 177
  5. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  6. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  7. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 108
  8. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  9. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 85
  10. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  11. 2023Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanationscited by 80
  12. 2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
38 more
  1. 2023Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision supportcited by 65
  2. 2024Calibrating Large Language Models Using Their Generations Onlycited by 63
  3. 2022Interactive Model Cards: A Human-Centered Approach to Model Documentationcited by 61
  4. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  5. 2024Determinants of LLM-assisted Decision-Makingcited by 55
  6. 2025Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistantcited by 54
  7. 2021Explanation-Based Human Debugging of NLP Models: A Surveycited by 53
  8. 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
  9. 2024Trust in AI: Progress, Challenges, and Future Directionscited by 35
  10. 2023A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspectivecited by 33
  11. 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
  12. 2023Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimizationcited by 26
  13. 2022Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Makingcited by 25
  14. 2022Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcitycited by 24
  15. 2021Algorithm Fairness in AI for Medicine and Healthcarecited by 21
  16. 2023A Systematic Review on Fostering Appropriate Trust in Human-AI Interactioncited by 21
  17. 2022Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and Engineeringcited by 20
  18. 2022Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Modelcited by 20
  19. 2024LLM-Generated Black-box Explanations Can Be Adversarially Helpfulcited by 20
  20. 2022"There Is Not Enough Information": On the Effects of Explanations on Perceptions of Informational Fairness and Trustworthiness in Automated Decision-Makingcited by 19
  21. 2024Correctness is not Faithfulness in RAG Attributionscited by 18
  22. 2021Learning to run a power network with trustcited by 17
  23. 2022Towards Explainable Evaluation Metrics for Natural Language Generationcited by 15
  24. 2023Impact Of Explainable AI On Cognitive Load: Insights From An Empirical Studycited by 15
  25. 2022Comparing Explanation Methods for Traditional Machine Learning Models Part 1: An Overview of Current Methods and Quantifying Their Disagreementcited by 14
  26. 2023A Review of the Role of Causality in Developing Trustworthy AI Systemscited by 14
  27. 2023Humans, AI, and Context: Understanding End-Users' Trust in a Real-World Computer Vision Applicationcited by 14
  28. 2024WWW: A Unified Framework for Explaining What, Where and Why of Neural Networks by Interpretation of Neuron Conceptscited by 14
  29. 2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14
  30. 2022Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficientcited by 13
  31. 2022Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methodscited by 13
  32. 2022VisFIS: Visual Feature Importance Supervision with Right-for-the-Right-Reason Objectivescited by 12
  33. 2025Forgetful but Faithful: A Cognitive Memory Architecture and Benchmark for Privacy-Aware Generative Agentscited by 12
  34. 2022A Dempster-Shafer approach to trustworthy AI with application to fetal brain MRI segmentationcited by 11
  35. 2022Are we measuring trust correctly in explainability, interpretability, and transparency research?cited by 11
  36. 2022Trust and Reliance in XAI -- Distinguishing Between Attitudinal and Behavioral Measurescited by 10
  37. 2021Intuitively Assessing ML Model Reliability through Example-Based Explanations and Editing Model Inputscited by 9
  38. 2022Towards a Roadmap on Software Engineering for Responsible AIcited by 9

50 most cited of 166 All 166 (JSON)