# How Does Conversation Length Impact User's Satisfaction? A Case Study of Length-Controlled Conversations with LLM-Powered Chatbots

arXiv: https://arxiv.org/abs/2404.17025 · HTML: https://mlsanity.com/paper/2404.17025/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2024.

## Built on by (most cited first, 8 of 8)

- [Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey](https://mlsanity.com/paper/2503.22458/) (2025)
- [MIRROR: Multimodal Cognitive Reframing Therapy for Rolling with Resistance](https://mlsanity.com/paper/2504.13211/) (2025)
- [OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models](https://mlsanity.com/paper/2508.21061/) (2025)
- [LLM Content Moderation and User Satisfaction: Evidence from Response Refusals in Chatbot Arena](https://mlsanity.com/paper/2501.03266/) (2025)
- [AI-exhibited Personality Traits Can Shape Human Self-concept through Conversations](https://mlsanity.com/paper/2601.12727/) (2026)
- [Does My Chatbot Have an Agenda? Understanding Human and AI Agency in Human-Human-like Chatbot Interaction](https://mlsanity.com/paper/2601.22452/) (2026)
- [Introducing the Swiss Food Knowledge Graph: AI for Context-Aware Nutrition Recommendation](https://mlsanity.com/paper/2507.10156/) (2025)
- [Pingquanqi (Equalizer): A Cross-Domain Sociotechnical Framework for Human-Agent Interaction Governance](https://mlsanity.com/paper/2606.26573/) (2026)

## Abstract (arXiv)

Users can discuss a wide range of topics with large language models (LLMs), but they do not always prefer solving problems or getting information through lengthy conversations. This raises an intriguing HCI question: How does instructing LLMs to engage in longer or shorter conversations affect conversation quality? In this paper, we developed two Slack chatbots using GPT-4 with the ability to vary conversation lengths and conducted a user study. Participants asked the chatbots both highly and less conversable questions, engaging in dialogues with 0, 3, 5, and 7 conversational turns. We found that the conversation quality does not differ drastically across different conditions, while participants had mixed reactions. Our study demonstrates LLMs' ability to change conversation length and the potential benefits for users resulting from such changes, but we caution that changes in text form may not necessarily imply changes in quality or content.