# The HCI Aspects of Public Deployment of Research Chatbots: A User Study, Design Recommendations, and Open Challenges

arXiv: https://arxiv.org/abs/2306.04765 · HTML: https://mlsanity.com/paper/2306.04765/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 31 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 8 of 8)

- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion](https://mlsanity.com/paper/2203.13224/) (2022)
- [ChatGPT: The End of Online Exam Integrity?](https://mlsanity.com/paper/2212.09292/) (2022)
- [Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents](https://mlsanity.com/paper/2201.04723/) (2022)
- [Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity](https://mlsanity.com/paper/2112.05843/) (2021)
- [Designing for Health Chatbots](https://mlsanity.com/paper/1902.09022/) (2019)

## Built on by (most cited first, 1 of 1)

- [Responsible Retrieval Augmented Generation for Climate Decision Making from Documents](https://mlsanity.com/paper/2410.23902/) (2024)

## Abstract (arXiv)

Publicly deploying research chatbots is a nuanced topic involving necessary risk-benefit analyses. While there have recently been frequent discussions on whether it is responsible to deploy such models, there has been far less focus on the interaction paradigms and design approaches that the resulting interfaces should adopt, in order to achieve their goals more effectively. We aim to pose, ground, and attempt to answer HCI questions involved in this scope, by reporting on a mixed-methods user study conducted on a recent research chatbot. We find that abstract anthropomorphic representation for the agent has a significant effect on user's perception, that offering AI explainability may have an impact on feedback rates, and that two (diegetic and extradiegetic) levels of the chat experience should be intentionally designed. We offer design recommendations and areas of further focus for the research community.