# Adversarial learning of neural user simulators for dialogue policy optimisation

arXiv: https://arxiv.org/abs/2306.00858 · HTML: https://mlsanity.com/paper/2306.00858/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 17 of 17)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adversarial Learning for Neural Dialogue Generation](https://mlsanity.com/paper/1701.06547/) (2017)
- [Adversarial Text-to-Image Synthesis: A Review](https://mlsanity.com/paper/2101.09983/) (2021)
- [Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systems](https://mlsanity.com/paper/1805.06966/) (2018)
- [A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems](https://mlsanity.com/paper/1607.00070/) (2016)
- [Transferable Dialogue Systems and User Simulators](https://mlsanity.com/paper/2107.11904/) (2021)
- [User Modeling for Task Oriented Dialogues](https://mlsanity.com/paper/1811.04369/) (2018)
- [Training language GANs from Scratch](https://mlsanity.com/paper/1905.09922/) (2019)
- [TextGAIL: Generative Adversarial Imitation Learning for Text Generation](https://mlsanity.com/paper/2004.13796/) (2020)
- [Image Synthesis with Adversarial Networks: a Comprehensive Survey and Case Studies](https://mlsanity.com/paper/2012.13736/) (2020)
- [Domain-independent User Simulation with Transformers for Task-oriented Dialogue Systems](https://mlsanity.com/paper/2106.08838/) (2021)
- [GenTUS: Simulating User Behaviour and Language in Task-oriented Dialogues with Generative Transformers](https://mlsanity.com/paper/2208.10817/) (2022)
- [Adversarial Learning of Task-Oriented Neural Dialog Models](https://mlsanity.com/paper/1805.11762/) (2018)
- [DAL: Dual Adversarial Learning for Dialogue Generation](https://mlsanity.com/paper/1906.09556/) (2019)
- [Multi-turn Dialogue Response Generation in an Adversarial Learning Framework](https://mlsanity.com/paper/1805.11752/) (2018)
- [Action State Update Approach to Dialogue Management](https://mlsanity.com/paper/2011.04637/) (2020)
- [Dialogue Strategy Adaptation to New Action Sets Using Multi-dimensional Modelling](https://mlsanity.com/paper/2204.07082/) (2022)

## Built on by (most cited first, 3 of 3)

- [Quantifying the Utility of User Simulators for Building Collaborative LLM Assistants](https://mlsanity.com/paper/2605.09808/) (2026)
- [DIAL: Direct Iterative Adversarial Learning for Realistic Multi-Turn Dialogue Simulation](https://mlsanity.com/paper/2512.20773/) (2025)
- [Stop Training Against Soft Users: Mentalized Adversarial Self-Play for Proactive Dialogue](https://mlsanity.com/paper/2610.21156/) (2026)

## Abstract (arXiv)

Reinforcement learning based dialogue policies are typically trained in interaction with a user simulator. To obtain an effective and robust policy, this simulator should generate user behaviour that is both realistic and varied. Current data-driven simulators are trained to accurately model the user behaviour in a dialogue corpus. We propose an alternative method using adversarial learning, with the aim to simulate realistic user behaviour with more variation. We train and evaluate several simulators on a corpus of restaurant search dialogues, and then use them to train dialogue system policies. In policy cross-evaluation experiments we demonstrate that an adversarially trained simulator produces policies with 8.3% higher success rate than those trained with a maximum likelihood simulator. Subjective results from a crowd-sourced dialogue system user evaluation confirm the effectiveness of adversarially training user simulators.