# Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue

arXiv: https://arxiv.org/abs/2009.06504 · HTML: https://mlsanity.com/paper/2009.06504/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2020.

## Built on by (most cited first, 26 of 26)

- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy](https://mlsanity.com/paper/2107.11823/) (2021)
- [Self- and Pseudo-self-supervised Prediction of Speaker and Key-utterance for Multi-party Dialogue Reading Comprehension](https://mlsanity.com/paper/2109.03772/) (2021)
- [Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehension](https://mlsanity.com/paper/2109.04066/) (2021)
- [Advances in Multi-turn Dialogue Comprehension: A Survey](https://mlsanity.com/paper/2103.03125/) (2021)
- [Exploring Dense Retrieval for Dialogue Response Selection](https://mlsanity.com/paper/2110.06612/) (2021)
- [Structural Characterization for Dialogue Disentanglement](https://mlsanity.com/paper/2110.08018/) (2021)
- [Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2204.08152/) (2022)
- [Hierarchical Dialogue Understanding with Special Tokens and Turn-level Attention](https://mlsanity.com/paper/2305.00262/) (2023)
- [Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems](https://mlsanity.com/paper/2106.01263/) (2021)
- [A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation](https://mlsanity.com/paper/2110.06823/) (2021)
- [Two-Level Supervised Contrastive Learning for Response Selection in Multi-Turn Dialogue](https://mlsanity.com/paper/2203.00793/) (2022)
- [Discourse-Aware Graph Networks for Textual Logical Reasoning](https://mlsanity.com/paper/2207.01450/) (2022)
- [Dynamically Retrieving Knowledge via Query Generation for Informative Dialogue Generation](https://mlsanity.com/paper/2208.00128/) (2022)
- [IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection](https://mlsanity.com/paper/2212.00482/) (2022)
- [Dramatic Conversation Disentanglement](https://mlsanity.com/paper/2305.16648/) (2023)
- [Learning Locality and Isotropy in Dialogue Modeling](https://mlsanity.com/paper/2205.14583/) (2022)
- [Generic Dependency Modeling for Multi-Party Conversation](https://mlsanity.com/paper/2302.10680/) (2023)
- [Intent-calibrated Self-training for Answer Selection in Open-domain Dialogues](https://mlsanity.com/paper/2307.06703/) (2023)
- [The Master-Slave Encoder Model for Improving Patent Text Summarization: A New Approach to Combining Specifications and Claims](https://mlsanity.com/paper/2411.14072/) (2024)
- [Advancing Multi-Party Dialogue Framework with Speaker-ware Contrastive Learning](https://mlsanity.com/paper/2501.11292/) (2025)
- [Advances in Multi-turn Dialogue Comprehension: A Survey](https://mlsanity.com/paper/2110.04984/) (2021)
- [STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension](https://mlsanity.com/paper/2212.12652/) (2022)
- [Channel-aware Decoupling Network for Multi-turn Dialogue Comprehension](https://mlsanity.com/paper/2301.03953/) (2023)
- [Coreference-aware Double-channel Attention Network for Multi-party Dialogue Reading Comprehension](https://mlsanity.com/paper/2305.08348/) (2023)
- [SPECTRUM: Speaker-Enhanced Pre-Training for Long Dialogue Summarization](https://mlsanity.com/paper/2401.17597/) (2024)

## Abstract (arXiv)

A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles. Thus utterance- and speaker-aware clues are supposed to be well captured in models. However, in the existing retrieval-based multi-turn dialogue modeling, the pre-trained language models (PrLMs) as encoder represent the dialogues coarsely by taking the pairwise dialogue history and candidate response as a whole, the hierarchical information on either utterance interrelation or speaker roles coupled in such representations is not well addressed. In this work, we propose a novel model to fill such a gap by modeling the effective utterance-aware and speaker-aware representations entailed in a dialogue history. In detail, we decouple the contextualized word representations by masking mechanisms in Transformer-based PrLM, making each word only focus on the words in current utterance, other utterances, two speaker roles (i.e., utterances of sender and utterances of receiver), respectively. Experimental results show that our method boosts the strong ELECTRA baseline substantially in four public benchmark datasets, and achieves various new state-of-the-art performance over previous methods. A series of ablation studies are conducted to demonstrate the effectiveness of our method.