# Non-Autoregressive Neural Dialogue Generation

arXiv: https://arxiv.org/abs/2002.04250 · HTML: https://mlsanity.com/paper/2002.04250/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 9 of 9)

- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [Transformer-based Conditional Variational Autoencoder for Controllable Story Generation](https://mlsanity.com/paper/2101.00828/) (2021)
- [Defending Against Backdoor Attacks in Natural Language Generation](https://mlsanity.com/paper/2106.01810/) (2021)
- [Thinking Clearly, Talking Fast: Concept-Guided Non-Autoregressive Generation for Open-Domain Dialogue Systems](https://mlsanity.com/paper/2109.04084/) (2021)
- [OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts](https://mlsanity.com/paper/2109.12761/) (2021)
- [NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2106.02210/) (2021)
- [SC2: Towards Enhancing Content Preservation and Style Consistency in Long Text Style Transfer](https://mlsanity.com/paper/2406.04578/) (2024)
- [Augmented Neural Fine-Tuning for Efficient Backdoor Purification](https://mlsanity.com/paper/2407.10052/) (2024)
- [RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformer](https://mlsanity.com/paper/2303.07665/) (2023)

## Abstract (arXiv)

Maximum Mutual information (MMI), which models the bidirectional dependency between responses (y) and contexts (x), i.e., the forward probability log p(y|x) and the backward probability log p(x|y), has been widely used as the objective in the sts model to address the dull-response issue in open-domain dialog generation. Unfortunately, under the framework of the sts model, direct decoding from log p(y|x) + log p(x|y) is infeasible since the second part (i.e., p(x|y)) requires the completion of target generation before it can be computed, and the search space for y is enormous. Empirically, an N-best list is first generated given p(y|x), and p(x|y) is then used to rerank the N-best list, which inevitably results in non-globally-optimal solutions. In this paper, we propose to use non-autoregressive (non-AR) generation model to address this non-global optimality issue. Since target tokens are generated independently in non-AR generation, p(x|y) for each target word can be computed as soon as it's generated, and does not have to wait for the completion of the whole sequence. This naturally resolves the non-global optimal issue in decoding. Experimental results demonstrate that the proposed non-AR strategy produces more diverse, coherent, and appropriate responses, yielding substantive gains in BLEU scores and in human evaluations.