# A User Simulator for Task-Completion Dialogues

arXiv: https://arxiv.org/abs/1612.05688 · HTML: https://mlsanity.com/paper/1612.05688/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2016.

## Built on by (most cited first, 30 of 57)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions](https://mlsanity.com/paper/2109.05794/) (2021)
- [Flipping the Dialogue: Training and Evaluating User Language Models](https://mlsanity.com/paper/2510.06552/) (2025)
- [Mind the Sim2Real Gap in User Simulation for Agentic Tasks](https://mlsanity.com/paper/2603.11245/) (2026)
- [MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Education](https://mlsanity.com/paper/2404.06711/) (2024)
- [Transferable Dialogue Systems and User Simulators](https://mlsanity.com/paper/2107.11904/) (2021)
- [Survey on reinforcement learning for language processing](https://mlsanity.com/paper/2104.05565/) (2021)
- [A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning](https://mlsanity.com/paper/2202.13675/) (2022)
- [Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2402.13374/) (2024)
- [Domain-independent User Simulation with Transformers for Task-oriented Dialogue Systems](https://mlsanity.com/paper/2106.08838/) (2021)
- [A Survey on Recent Advances in Conversational Data Generation](https://mlsanity.com/paper/2405.13003/) (2024)
- [Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning](https://mlsanity.com/paper/1801.06176/) (2018)
- [Goal Alignment in LLM-Based User Simulators for Conversational AI](https://mlsanity.com/paper/2507.20152/) (2025)
- [Learning to Simulate Natural Language Feedback for Interactive Semantic Parsing](https://mlsanity.com/paper/2305.08195/) (2023)
- [LAD: Language Models as Data for Zero-Shot Dialog](https://mlsanity.com/paper/2207.14393/) (2022)
- [NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation](https://mlsanity.com/paper/2105.14454/) (2021)
- [Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator](https://mlsanity.com/paper/2210.14529/) (2022)
- [Social Influence Dialogue Systems: A Survey of Datasets and Models For Social Influence Tasks](https://mlsanity.com/paper/2210.05664/) (2022)
- [Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning](https://mlsanity.com/paper/1811.07550/) (2018)
- [Injecting Salesperson's Dialogue Strategies in Large Language Models with Chain-of-Thought Reasoning](https://mlsanity.com/paper/2404.18564/) (2024)
- [Anti-Overestimation Dialogue Policy Learning for Task-Completion Dialogue System](https://mlsanity.com/paper/2207.11762/) (2022)
- [DEUS: A Data-driven Approach to Estimate User Satisfaction in Multi-turn Dialogues](https://mlsanity.com/paper/2103.01287/) (2021)
- [SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue](https://mlsanity.com/paper/2602.03548/) (2026)
- [Teaching Models new APIs: Domain-Agnostic Simulators for Task Oriented Dialogue](https://mlsanity.com/paper/2110.06905/) (2021)
- [The Behavior Gap: Evaluating Zero-shot LLM Agents in Complex Task-Oriented Dialogs](https://mlsanity.com/paper/2506.12266/) (2025)
- [Multi-trait User Simulation with Adaptive Decoding for Conversational Task Assistants](https://mlsanity.com/paper/2410.12891/) (2024)
- [Interactive Video Corpus Moment Retrieval using Reinforcement Learning](https://mlsanity.com/paper/2302.09522/) (2023)
- [Reinforcing Human Behavior Simulation via Verbal Feedback](https://mlsanity.com/paper/2605.20506/) (2026)
- [High-Quality Diversification for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2106.00891/) (2021)
- [Are Current Task-oriented Dialogue Systems Able to Satisfy Impolite Users?](https://mlsanity.com/paper/2210.12942/) (2022)

## Abstract (arXiv)

Despite widespread interests in reinforcement-learning for task-oriented dialogue systems, several obstacles can frustrate research and development progress. First, reinforcement learners typically require interaction with the environment, so conventional dialogue corpora cannot be used directly. Second, each task presents specific challenges, requiring separate corpus of task-specific annotated data. Third, collecting and annotating human-machine or human-human conversations for task-oriented dialogues requires extensive domain knowledge. Because building an appropriate dataset can be both financially costly and time-consuming, one popular approach is to build a user simulator based upon a corpus of example dialogues. Then, one can train reinforcement learning agents in an online fashion as they interact with the simulator. Dialogue agents trained on these simulators can serve as an effective starting point. Once agents master the simulator, they may be deployed in a real environment to interact with humans, and continue to be trained online. To ease empirical algorithmic comparisons in dialogues, this paper introduces a new, publicly available simulation framework, where our simulator, designed for the movie-booking domain, leverages both rules and collected data. The simulator supports two tasks: movie ticket booking and movie seeking. Finally, we demonstrate several agents and detail the procedure to add and test your own agent in the proposed framework.