# Plan Explanations as Model Reconciliation -- An Empirical Study

arXiv: https://arxiv.org/abs/1802.01013 · HTML: https://mlsanity.com/paper/1802.01013/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2018.

## Built on by (most cited first, 7 of 7)

- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [Contrastive Explanations of Plans Through Model Restrictions](https://mlsanity.com/paper/2103.15575/) (2021)
- [A Framework for Inherently Interpretable Optimization Models](https://mlsanity.com/paper/2208.12570/) (2022)
- [An Information Bottleneck Characterization of the Understanding-Workload Tradeoff](https://mlsanity.com/paper/2310.07802/) (2023)
- [One-shot Policy Elicitation via Semantic Reward Manipulation](https://mlsanity.com/paper/2101.01860/) (2021)
- [Defining Explainable AI for Requirements Analysis](https://mlsanity.com/paper/2602.19071/) (2026)

## Abstract (arXiv)

Recent work in explanation generation for decision making agents has looked at how unexplained behavior of autonomous systems can be understood in terms of differences in the model of the system and the human's understanding of the same, and how the explanation process as a result of this mismatch can be then seen as a process of reconciliation of these models. Existing algorithms in such settings, while having been built on contrastive, selective and social properties of explanations as studied extensively in the psychology literature, have not, to the best of our knowledge, been evaluated in settings with actual humans in the loop. As such, the applicability of such explanations to human-AI and human-robot interactions remains suspect. In this paper, we set out to evaluate these explanation generation algorithms in a series of studies in a mock search and rescue scenario with an internal semi-autonomous robot and an external human commander. We demonstrate to what extent the properties of these algorithms hold as they are evaluated by humans, and how the dynamics of trust between the human and the robot evolve during the process of these interactions.