# PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism

arXiv: https://arxiv.org/abs/2212.09086 · HTML: https://mlsanity.com/paper/2212.09086/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 58 references that are in the map.
- First posted on arXiv 18 Dec 2022.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders](https://mlsanity.com/paper/1703.10960/) (2017)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues](https://mlsanity.com/paper/1605.06069/) (2016)
- [PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable](https://mlsanity.com/paper/1910.07931/) (2019)
- [Neural Responding Machine for Short-Text Conversation](https://mlsanity.com/paper/1503.02364/) (2015)
- [Audio-Visual Scene-Aware Dialog](https://mlsanity.com/paper/1901.09107/) (2019)
- [Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse](https://mlsanity.com/paper/1911.02469/) (2019)
- [Adversarial Learning for Neural Dialogue Generation](https://mlsanity.com/paper/1701.06547/) (2017)
- [BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data](https://mlsanity.com/paper/2106.06169/) (2021)
- [DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder](https://mlsanity.com/paper/1805.12352/) (2018)
- [DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text](https://mlsanity.com/paper/1802.01345/) (2018)
- [Variational Transformers for Diverse Response Generation](https://mlsanity.com/paper/2003.12738/) (2020)
- [DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation](https://mlsanity.com/paper/2204.13031/) (2022)
- [Hierarchical Recurrent Attention Network for Response Generation](https://mlsanity.com/paper/1701.07149/) (2017)
- [Unsupervised Discrete Sentence Representation Learning for Interpretable Neural Dialog Generation](https://mlsanity.com/paper/1804.08069/) (2018)
- [Variational Attention for Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.08207/) (2017)
- [Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity](https://mlsanity.com/paper/1809.06873/) (2018)
- [Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task](https://mlsanity.com/paper/1806.03578/) (2018)
- [Detecting Regions of Maximal Divergence for Spatio-Temporal Anomaly Detection](https://mlsanity.com/paper/1804.07091/) (2018)
- [Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders](https://mlsanity.com/paper/2106.03410/) (2021)

## Built on by (most cited first, 6 of 6)

- [Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding](https://mlsanity.com/paper/2307.07880/) (2023)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)
- [Evaluate What You Can't Evaluate: Unassessable Quality for Generated Response](https://mlsanity.com/paper/2305.14658/) (2023)
- [It Couldn't Help But Overhear: On the Limits of Modelling Meta-Communicative Grounding Acts with Supervised Learning](https://mlsanity.com/paper/2405.01139/) (2024)
- [A Unified Data Augmentation Framework for Low-Resource Multi-Domain Dialogue Generation](https://mlsanity.com/paper/2406.09881/) (2024)
- [ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Language](https://mlsanity.com/paper/2408.08724/) (2024)

## Abstract (arXiv)

We investigate response generation for multi-turn dialogue in generative-based chatbots. Existing generative models based on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the sequences, which makes models unable to capture the subtle variability observed in different dialogues and cannot distinguish the differences between dialogues that are similar in composition. In this paper, we propose a Pseudo-Variational Gated Recurrent Unit (PVGRU) component without posterior knowledge through introducing a recurrent summarizing variable into the GRU, which can aggregate the accumulated distribution variations of subsequences. PVGRU can perceive the subtle semantic variability through summarizing variables that are optimized by the devised distribution consistency and reconstruction objectives. In addition, we build a Pseudo-Variational Hierarchical Dialogue (PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity and relevance of responses on two benchmark datasets.