# A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation

arXiv: https://arxiv.org/abs/2110.06823 · HTML: https://mlsanity.com/paper/2110.06823/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 13 Oct 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [A Persona-Based Neural Conversation Model](https://mlsanity.com/paper/1603.06155/) (2016)
- [Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models](https://mlsanity.com/paper/1507.04808/) (2015)
- [A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues](https://mlsanity.com/paper/1605.06069/) (2016)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable](https://mlsanity.com/paper/1910.07931/) (2019)
- [PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning](https://mlsanity.com/paper/2006.16779/) (2020)
- [Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/1804.08217/) (2018)
- [A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestion](https://mlsanity.com/paper/1507.02221/) (2015)
- [Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study](https://mlsanity.com/paper/1906.01603/) (2019)
- [Hierarchical Recurrent Attention Network for Response Generation](https://mlsanity.com/paper/1701.07149/) (2017)
- [Data Manipulation: Towards Effective Instance Learning for Neural Dialogue Generation via Learning to Augment and Reweight](https://mlsanity.com/paper/2004.02594/) (2020)
- [ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation](https://mlsanity.com/paper/1907.05339/) (2019)
- [Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning](https://mlsanity.com/paper/1606.02560/) (2016)
- [Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue](https://mlsanity.com/paper/2009.06504/) (2020)
- [Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders](https://mlsanity.com/paper/2106.03410/) (2021)
- [Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks](https://mlsanity.com/paper/2004.01972/) (2020)
- [Towards Neural Speaker Modeling in Multi-Party Conversation: The Task, Dataset, and Models](https://mlsanity.com/paper/1708.03152/) (2017)
- [Generating Multiple Diverse Responses with Multi-Mapping and Posterior Mapping Selection](https://mlsanity.com/paper/1906.01781/) (2019)
- [Implicit Discourse Relation Identification for Open-domain Dialogues](https://mlsanity.com/paper/1907.03975/) (2019)
- [Speaker Role Contextual Modeling for Language Understanding and Dialogue Policy Learning](https://mlsanity.com/paper/1710.00164/) (2017)
- [A Persona-based Multi-turn Conversation Model in an Adversarial Learning Framework](https://mlsanity.com/paper/1905.01998/) (2019)

## Built on by (most cited first, 2 of 2)

- [Persona-aware Generative Model for Code-mixed Language](https://mlsanity.com/paper/2309.02915/) (2023)
- [Learning states enhanced knowledge tracing: Simulating the diversity in real-world learning process](https://mlsanity.com/paper/2412.19550/) (2024)

## Abstract (arXiv)

This paper presents a novel open-domain dialogue generation model emphasizing the differentiation of speakers in multi-turn conversations. Differing from prior work that solely relies on the content of conversation history to generate a response, we argue that capturing relative social relations among utterances (i.e., generated by either the same speaker or different persons) benefits the machine capturing fine-grained context information from a conversation history to improve context coherence in the generated response. Given that, we propose a speaker-aware Parallel Hierarchical Attentive Encoder-Decoder (PHAED) model that aims to model each utterance with the awareness of its speaker and contextual associations with the same speaker's previous messages. Specifically, in a conversation involving two speakers, we regard the utterances from one speaker as responses and those from the other as queries. After understanding queries via our encoder with inner-query and inter-query encodings, our decoder reuses the hidden states of previously generated responses, instead of reconstructing these by the encoder, to generate a new response. Our empirical results show that PHAED outperforms the state-of-the-art in both automatic and human evaluations. Furthermore, our ablation study shows that dialogue models with speaker tokens can generally decrease the possibility of generating non-coherent responses regarding the conversation context.