# User Intent Recognition and Satisfaction with Large Language Models: A User Study with ChatGPT

arXiv: https://arxiv.org/abs/2402.02136 · HTML: https://mlsanity.com/paper/2402.02136/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2024.

## Built on by (most cited first, 19 of 19)

- [Emergent Abilities in Large Language Models: A Survey](https://mlsanity.com/paper/2503.05788/) (2025)
- [Thought-Augmented Planning for LLM-Powered Interactive Recommender Agent](https://mlsanity.com/paper/2506.23485/) (2025)
- [A User-Centric Multi-Intent Benchmark for Evaluating Large Language Models](https://mlsanity.com/paper/2404.13940/) (2024)
- [Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery](https://mlsanity.com/paper/2504.01205/) (2025)
- [Exploring Prompt Engineering Practices in the Enterprise](https://mlsanity.com/paper/2403.08950/) (2024)
- [Translation in the Hands of Many:Centering Lay Users in Machine Translation Interactions](https://mlsanity.com/paper/2502.13780/) (2025)
- [Modeling Challenging Patient Interactions: LLMs for Medical Communication Training](https://mlsanity.com/paper/2503.22250/) (2025)
- [Toward Super Agent System with Hybrid AI Routers](https://mlsanity.com/paper/2504.10519/) (2025)
- [Know Your Intent: An Autonomous Multi-Perspective LLM Agent Framework for DeFi User Transaction Intent Mining](https://mlsanity.com/paper/2511.15456/) (2025)
- [SING: Synthetic Intention Graph for Scalable Active Tool Discovery in LLM Agents](https://mlsanity.com/paper/2606.16591/) (2026)
- [From Passive Watching to Active Learning: Empowering Proactive Participation in Digital Classrooms with AI Video Assistant](https://mlsanity.com/paper/2409.15843/) (2024)
- [Trapped by Expectations: Functional Fixedness in LLM-Enabled Chat Search](https://mlsanity.com/paper/2504.02074/) (2025)
- [A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges](https://mlsanity.com/paper/2507.21117/) (2025)
- [Is Crowdsourcing Breaking Your Bank? Cost-Effective Fine-Tuning of Pre-trained Language Models with Proximal Policy Optimization](https://mlsanity.com/paper/2402.18284/) (2024)
- [Review of Hallucination Understanding in Large Language and Vision Models](https://mlsanity.com/paper/2510.00034/) (2025)
- [Generative AI Practices, Literacy, and Divides: An Empirical Analysis in the Italian Context](https://mlsanity.com/paper/2512.03671/) (2025)
- [Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answering](https://mlsanity.com/paper/2605.12645/) (2026)
- [SCISSOR: Mitigating Semantic Bias through Cluster-Aware Siamese Networks for Robust Classification](https://mlsanity.com/paper/2506.14587/) (2025)
- [IntPro: A Proxy Agent for Context-Aware Intent Understanding via Retrieval-conditioned Inference](https://mlsanity.com/paper/2603.03325/) (2026)

## Abstract (arXiv)

The rapid evolution of LLMs represents an impactful paradigm shift in digital interaction and content engagement. While they encode vast amounts of human-generated knowledge and excel in processing diverse data types, they often face the challenge of accurately responding to specific user intents, leading to user dissatisfaction. Based on a fine-grained intent taxonomy and intent-based prompt reformulations, we analyze the quality of intent recognition and user satisfaction with answers from intent-based prompt reformulations of GPT-3.5 Turbo and GPT-4 Turbo models. Our study highlights the importance of human-AI interaction and underscores the need for interdisciplinary approaches to improve conversational AI systems. We show that GPT-4 outperforms GPT-3.5 in recognizing common intents but is often outperformed by GPT-3.5 in recognizing less frequent intents. Moreover, whenever the user intent is correctly recognized, while users are more satisfied with the intent-based reformulations of GPT-4 compared to GPT-3.5, they tend to be more satisfied with the models' answers to their original prompts compared to the reformulated ones. The collected data from our study has been made publicly available on GitHub (https://github.com/ConcealedIDentity/UserIntentStudy) for further research.