# Investigating the effect of Mental Models in User Interaction with an Adaptive Dialog Agent

arXiv: https://arxiv.org/abs/2408.14154 · HTML: https://mlsanity.com/paper/2408.14154/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 42 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 6 of 6)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [Conversational Tree Search: A New Hybrid Dialog Task](https://mlsanity.com/paper/2303.10227/) (2023)
- [Understanding Mental Models of AI through Player-AI Interaction](https://mlsanity.com/paper/2103.16168/) (2021)
- [Towards a Zero-Data, Controllable, Adaptive Dialog System](https://mlsanity.com/paper/2403.17582/) (2024)

## Abstract (arXiv)

Mental models play an important role in whether user interaction with intelligent systems, such as dialog systems is successful or not. Adaptive dialog systems present the opportunity to align a dialog agent's behavior with heterogeneous user expectations. However, there has been little research into what mental models users form when interacting with a task-oriented dialog system, how these models affect users' interactions, or what role system adaptation can play in this process, making it challenging to avoid damage to human-AI partnership. In this work, we collect a new publicly available dataset for exploring user mental models about information seeking dialog systems. We demonstrate that users have a variety of conflicting mental models about such systems, the validity of which directly impacts the success of their interactions and perceived usability of system. Furthermore, we show that adapting a dialog agent's behavior to better align with users' mental models, even when done implicitly, can improve perceived usability, dialog efficiency, and success. To this end, we argue that implicit adaptation can be a valid strategy for task-oriented dialog systems, so long as developers first have a solid understanding of users' mental models.