# X-ToM: Explaining with Theory-of-Mind for Gaining Justified Human Trust

arXiv: https://arxiv.org/abs/1909.06907 · HTML: https://mlsanity.com/paper/1909.06907/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2019.

## Built on by (most cited first, 8 of 8)

- [Explanations in Autonomous Driving: A Survey](https://mlsanity.com/paper/2103.05154/) (2021)
- [CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models](https://mlsanity.com/paper/2109.01401/) (2021)
- [Towards Socially Intelligent Agents with Mental State Transition and Human Utility](https://mlsanity.com/paper/2103.07011/) (2021)
- [Attention cannot be an Explanation](https://mlsanity.com/paper/2201.11194/) (2022)
- [Question Generation for Evaluating Cross-Dataset Shifts in Multi-modal Grounding](https://mlsanity.com/paper/2201.09639/) (2022)
- [Discourse Analysis for Evaluating Coherence in Video Paragraph Captions](https://mlsanity.com/paper/2201.06207/) (2022)
- [A formal definition and meta-model for a machine theory of mind](https://mlsanity.com/paper/2606.03471/) (2026)
- [On the Importance of Karaka Framework in Multi-modal Grounding](https://mlsanity.com/paper/2204.04347/) (2022)

## Abstract (arXiv)

We present a new explainable AI (XAI) framework aimed at increasing justified human trust and reliance in the AI machine through explanations. We pose explanation as an iterative communication process, i.e. dialog, between the machine and human user. More concretely, the machine generates sequence of explanations in a dialog which takes into account three important aspects at each dialog turn: (a) human's intention (or curiosity); (b) human's understanding of the machine; and (c) machine's understanding of the human user. To do this, we use Theory of Mind (ToM) which helps us in explicitly modeling human's intention, machine's mind as inferred by the human as well as human's mind as inferred by the machine. In other words, these explicit mental representations in ToM are incorporated to learn an optimal explanation policy that takes into account human's perception and beliefs. Furthermore, we also show that ToM facilitates in quantitatively measuring justified human trust in the machine by comparing all the three mental representations. We applied our framework to three visual recognition tasks, namely, image classification, action recognition, and human body pose estimation. We argue that our ToM based explanations are practical and more natural for both expert and non-expert users to understand the internal workings of complex machine learning models. To the best of our knowledge, this is the first work to derive explanations using ToM. Extensive human study experiments verify our hypotheses, showing that the proposed explanations significantly outperform the state-of-the-art XAI methods in terms of all the standard quantitative and qualitative XAI evaluation metrics including human trust, reliance, and explanation satisfaction.