# Learning from Perturbations: Diverse and Informative Dialogue Generation with Inverse Adversarial Training

arXiv: https://arxiv.org/abs/2105.15171 · HTML: https://mlsanity.com/paper/2105.15171/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 26 references that are in the map.
- First posted on arXiv 31 May 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders](https://mlsanity.com/paper/1703.10960/) (2017)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization](https://mlsanity.com/paper/1809.05972/) (2018)
- [Learning End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1605.07683/) (2016)
- [Adversarial Learning for Neural Dialogue Generation](https://mlsanity.com/paper/1701.06547/) (2017)
- [A Neural Network Approach to Context-Sensitive Generation of Conversational Responses](https://mlsanity.com/paper/1506.06714/) (2015)
- [Reward Augmented Maximum Likelihood for Neural Structured Prediction](https://mlsanity.com/paper/1609.00150/) (2016)
- [Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study](https://mlsanity.com/paper/1906.01603/) (2019)
- [Syntax-guided Controlled Generation of Paraphrases](https://mlsanity.com/paper/2005.08417/) (2020)
- [DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text](https://mlsanity.com/paper/1802.01345/) (2018)
- [Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints](https://mlsanity.com/paper/1809.01215/) (2018)
- [Improving Neural Conversational Models with Entropy-Based Data Filtering](https://mlsanity.com/paper/1905.05471/) (2019)
- [Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models](https://mlsanity.com/paper/2002.05058/) (2020)
- [Self-Adversarial Learning with Comparative Discrimination for Text Generation](https://mlsanity.com/paper/2001.11691/) (2020)
- [DAL: Dual Adversarial Learning for Dialogue Generation](https://mlsanity.com/paper/1906.09556/) (2019)

## Built on by (most cited first, 13 of 13)

- [Recent Advances in Neural Text Generation: A Task-Agnostic Survey](https://mlsanity.com/paper/2203.03047/) (2022)
- [Factual and Informative Review Generation for Explainable Recommendation](https://mlsanity.com/paper/2209.12613/) (2022)
- [An Empirical Study on the Overlapping Problem of Open-Domain Dialogue Datasets](https://mlsanity.com/paper/2201.06219/) (2022)
- [Diversifying Neural Dialogue Generation via Negative Distillation](https://mlsanity.com/paper/2205.02795/) (2022)
- [Diversifying Question Generation over Knowledge Base via External Natural Questions](https://mlsanity.com/paper/2309.14362/) (2023)
- [Does Incomplete Syntax Influence Korean Language Model? Focusing on Word Order and Case Markers](https://mlsanity.com/paper/2407.09184/) (2024)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)
- [Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention](https://mlsanity.com/paper/2211.07164/) (2022)
- [Sequential Topic Selection Model with Latent Variable for Topic-Grounded Dialogue](https://mlsanity.com/paper/2210.08801/) (2022)
- [Neural Conversation Models and How to Rein Them in: A Survey of Failures and Fixes](https://mlsanity.com/paper/2308.06095/) (2023)
- [Penalizing Confident Predictions on Largely Perturbed Inputs Does Not Improve Out-of-Distribution Generalization in Question Answering](https://mlsanity.com/paper/2211.16093/) (2022)
- [Can Perturbations Help Reduce Investment Risks? Risk-Aware Stock Recommendation via Split Variational Adversarial Training](https://mlsanity.com/paper/2304.11043/) (2023)
- [Exploring the Impact of Human Evaluator Group on Chat-Oriented Dialogue Evaluation](https://mlsanity.com/paper/2309.07998/) (2023)

## Abstract (arXiv)

In this paper, we propose Inverse Adversarial Training (IAT) algorithm for training neural dialogue systems to avoid generic responses and model dialogue history better. In contrast to standard adversarial training algorithms, IAT encourages the model to be sensitive to the perturbation in the dialogue history and therefore learning from perturbations. By giving higher rewards for responses whose output probability reduces more significantly when dialogue history is perturbed, the model is encouraged to generate more diverse and consistent responses. By penalizing the model when generating the same response given perturbed dialogue history, the model is forced to better capture dialogue history and generate more informative responses. Experimental results on two benchmark datasets show that our approach can better model dialogue history and generate more diverse and consistent responses. In addition, we point out a problem of the widely used maximum mutual information (MMI) based methods for improving the diversity of dialogue response generation models and demonstrate it empirically.