# Simulating User Diversity in Task-Oriented Dialogue Systems using Large Language Models

arXiv: https://arxiv.org/abs/2502.12813 · HTML: https://mlsanity.com/paper/2502.12813/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 9 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Builds on (most cited first, 6 of 6)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Are LLMs All You Need for Task-Oriented Dialogue?](https://mlsanity.com/paper/2304.06556/) (2023)
- [Synthetic Dialogue Dataset Generation using LLM Agents](https://mlsanity.com/paper/2401.17461/) (2024)
- [User Modeling for Task Oriented Dialogues](https://mlsanity.com/paper/1811.04369/) (2018)

## Built on by (most cited first, 1 of 1)

- [Agentic Persona Control and Task State Tracking for Realistic User Simulation in Interactive Scenarios](https://mlsanity.com/paper/2601.15290/) (2025)

## Abstract (arXiv)

In this study, we explore the application of Large Language Models (LLMs) for generating synthetic users and simulating user conversations with a task-oriented dialogue system and present detailed results and their analysis. We propose a comprehensive novel approach to user simulation technique that uses LLMs to create diverse user profiles, set goals, engage in multi-turn dialogues, and evaluate the conversation success. We employ two proprietary LLMs, namely GPT-4o and GPT-o1 (Achiam et al., 2023), to generate a heterogeneous base of user profiles, characterized by varied demographics, multiple user goals, different conversational styles, initial knowledge levels, interests, and conversational objectives. We perform a detailed analysis of the user profiles generated by LLMs to assess the diversity, consistency, and potential biases inherent in these LLM-generated user simulations. We find that GPT-o1 generates more heterogeneous user distribution across most user attributes, while GPT-4o generates more skewed user attributes. The generated set of user profiles are then utilized to simulate dialogue sessions by interacting with a task-oriented dialogue system.