# Adversarial Learning for Neural Dialogue Generation

arXiv: https://arxiv.org/abs/1701.06547 · HTML: https://mlsanity.com/paper/1701.06547/

## Key facts

- Cited by 103 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2017.

## Built on by (most cited first, 30 of 103)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog](https://mlsanity.com/paper/1907.00456/) (2019)
- [User Behavior Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2306.02552/) (2023)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Human-centric Dialog Training via Offline Reinforcement Learning](https://mlsanity.com/paper/2010.05848/) (2020)
- [Text Generation by Learning from Demonstrations](https://mlsanity.com/paper/2009.07839/) (2020)
- [MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation](https://mlsanity.com/paper/2211.05719/) (2022)
- [Multi-Agent Imitation Learning for Driving Simulation](https://mlsanity.com/paper/1803.01044/) (2018)
- [Defending Against Backdoor Attacks in Natural Language Generation](https://mlsanity.com/paper/2106.01810/) (2021)
- [Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View](https://mlsanity.com/paper/2203.12258/) (2022)
- [OTTers: One-turn Topic Transitions for Open-Domain Dialogue](https://mlsanity.com/paper/2105.13710/) (2021)
- [A Literature Survey of Recent Advances in Chatbots](https://mlsanity.com/paper/2201.06657/) (2022)
- [AI-Generated Content (AIGC) for Various Data Modalities: A Survey](https://mlsanity.com/paper/2308.14177/) (2023)
- [Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning](https://mlsanity.com/paper/1812.03509/) (2018)
- [Why is constrained neural language generation particularly challenging?](https://mlsanity.com/paper/2206.05395/) (2022)
- [ATM: Adversarial Tuning Multi-agent System Makes a Robust Retrieval-Augmented Generator](https://mlsanity.com/paper/2405.18111/) (2024)
- [Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders](https://mlsanity.com/paper/2106.03410/) (2021)
- [Speaker Turn Modeling for Dialogue Act Classification](https://mlsanity.com/paper/2109.05056/) (2021)
- [Toward Human-Like Evaluation for Natural Language Generation with Error Analysis](https://mlsanity.com/paper/2212.10179/) (2022)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts](https://mlsanity.com/paper/2109.12761/) (2021)
- [A Mixture-of-Expert Approach to RL-based Dialogue Management](https://mlsanity.com/paper/2206.00059/) (2022)
- [Measuring and Improving Semantic Diversity of Dialogue Generation](https://mlsanity.com/paper/2210.05725/) (2022)
- [Time-Series Regeneration with Convolutional Recurrent Generative Adversarial Network for Remaining Useful Life Estimation](https://mlsanity.com/paper/2101.03678/) (2021)
- [Row Conditional-TGAN for generating synthetic relational databases](https://mlsanity.com/paper/2211.07588/) (2022)
- [Adaptive Social Learning via Mode Policy Optimization for Language Agents](https://mlsanity.com/paper/2505.02156/) (2025)
- [InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Model](https://mlsanity.com/paper/2102.11008/) (2021)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs](https://mlsanity.com/paper/2106.06363/) (2021)
- [Controllable and Diverse Text Generation in E-commerce](https://mlsanity.com/paper/2102.11497/) (2021)

## Abstract (arXiv)

In this paper, drawing intuition from the Turing test, we propose using adversarial training for open-domain dialogue generation: the system is trained to produce sequences that are indistinguishable from human-generated dialogue utterances. We cast the task as a reinforcement learning (RL) problem where we jointly train two systems, a generative model to produce response sequences, and a discriminator---analagous to the human evaluator in the Turing test--- to distinguish between the human-generated dialogues and the machine-generated ones. The outputs from the discriminator are then used as rewards for the generative model, pushing the system to generate dialogues that mostly resemble human dialogues. In addition to adversarial training we describe a model for adversarial em evaluation that uses success in fooling an adversary as a dialogue evaluation metric, while avoiding a number of potential pitfalls. Experimental results on several metrics, including adversarial evaluation, demonstrate that the adversarially-trained system generates higher-quality responses than previous baselines.