DialoGen: Generalized Long-Range Context Representation for Dialogue Systems
Not on the map. Why?
Abstract (arXiv)
Long-range context modeling is crucial to both dialogue understanding and generation. The most popular method for dialogue context representation is to concatenate the last-k utterances in chronological order. However, this method may not be ideal for conversations containing long-range dependencies, i.e., when there is a need to look beyond last-k utterances to generate a meaningful response. In this work, we propose DialoGen, a novel encoder-decoder based framework for dialogue generation with a generalized context representation that can look beyond the last-k utterances. The main idea of our approach is to identify and utilize the most relevant historical utterances instead of last-k, which also enables the compact representation of dialogue history with fewer tokens. We study the effectiveness of our proposed method on both dialogue generation (open-domain) and understanding (DST). Even with a compact context representation, DialoGen performs comparably to the state-of-the-art models on the open-domain DailyDialog dataset. We observe a similar behavior on the DST task of the MultiWOZ dataset when the proposed context representation is applied to existing DST models. We also discuss the generalizability and interpretability of DialoGen and show that the relevance score of previous utterances agrees well with human cognition.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 12 Oct 2022.
Builds on
31 of 49 references.
References · 31
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2016Layer Normalizationcited by 5,579
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2020Big Bird: Transformers for Longer Sequencescited by 1,673
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
- 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
- 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
- 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
- 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
19 more
- 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
- 2016A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialoguescited by 174
- 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
- 2020TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Trackingcited by 130
- 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 114
- 2015A Neural Network Approach to Context-Sensitive Generation of Conversational Responsescited by 98
- 2021A Comprehensive Assessment of Dialog Evaluation Metricscited by 90
- 2020Sequential Latent Knowledge Selection for Knowledge-Grounded Dialoguecited by 82
- 2019Efficient Dialogue State Tracking by Selectively Overwriting Memorycited by 76
- 2015A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestioncited by 68
- 2021Poolingformer: Long Document Modeling with Pooling Attentioncited by 60
- 2019Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple Referencescited by 48
- 2021Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterancescited by 40
- 2020DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterancescited by 36
- 2019Incremental Transformer with Deliberation Decoder for Document Grounded Conversationscited by 35
- 2022DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generationcited by 35
- 2017Hierarchical Recurrent Attention Network for Response Generationcited by 34
- 2019ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generationcited by 32
- 2020Mask & Focus: Conversation Modelling by Learning Conceptscited by 1
Built on by
Citing papers per year: 2025: 2.
2 papers.