# Explainable Human-AI Interaction: A Planning Perspective

arXiv: https://arxiv.org/abs/2405.15804 · HTML: https://mlsanity.com/paper/2405.15804/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 93 references that are in the map.
- First posted on arXiv 19 May 2024.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Graying the black box: Understanding DQNs](https://mlsanity.com/paper/1602.02658/) (2016)
- [Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences](https://mlsanity.com/paper/1807.08706/) (2018)
- [Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation](https://mlsanity.com/paper/2006.14804/) (2020)
- [Bridging the Gap: Providing Post-Hoc Symbolic Explanations for Sequential Decision-Making Problems with Inscrutable Representations](https://mlsanity.com/paper/2002.01080/) (2020)
- [Planning for Goal-Oriented Dialogue Systems](https://mlsanity.com/paper/1910.08137/) (2019)
- [Signaling Friends and Head-Faking Enemies Simultaneously: Balancing Goal Obfuscation and Goal Legibility](https://mlsanity.com/paper/1905.10672/) (2019)
- [Robot Planning with Mathematical Models of Human State and Action](https://mlsanity.com/paper/1705.04226/) (2017)
- [Not all users are the same: Providing personalized explanations for sequential decision making problems](https://mlsanity.com/paper/2106.12207/) (2021)
- [Learning from Ambiguous Demonstrations with Self-Explanation Guided Reinforcement Learning](https://mlsanity.com/paper/2110.05286/) (2021)
- [Model Elicitation through Direct Questioning](https://mlsanity.com/paper/2011.12262/) (2020)
- [Trust-Aware Planning: Modeling Trust Evolution in Iterated Human-Robot Interaction](https://mlsanity.com/paper/2105.01220/) (2021)

## Abstract (arXiv)

From its inception, AI has had a rather ambivalent relationship with humans -- swinging between their augmentation and replacement. Now, as AI technologies enter our everyday lives at an ever increasing pace, there is a greater need for AI systems to work synergistically with humans. One critical requirement for such synergistic human-AI interaction is that the AI systems be explainable to the humans in the loop. To do this effectively, AI agents need to go beyond planning with their own models of the world, and take into account the mental model of the human in the loop. Drawing from several years of research in our lab, we will discuss how the AI agent can use these mental models to either conform to human expectations, or change those expectations through explanatory communication. While the main focus of the book is on cooperative scenarios, we will point out how the same mental models can be used for obfuscation and deception. Although the book is primarily driven by our own research in these areas, in every chapter, we will provide ample connections to relevant research from other groups.