# Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer

arXiv: https://arxiv.org/abs/1911.10666 · HTML: https://mlsanity.com/paper/1911.10666/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2019.

## Built on by (most cited first, 16 of 16)

- [Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues](https://mlsanity.com/paper/2103.00820/) (2021)
- [Unsupervised Conversation Disentanglement through Co-Training](https://mlsanity.com/paper/2109.03199/) (2021)
- [Structural Characterization for Dialogue Disentanglement](https://mlsanity.com/paper/2110.08018/) (2021)
- [Findings on Conversation Disentanglement](https://mlsanity.com/paper/2112.05346/) (2021)
- [Conversation Disentanglement with Bi-Level Contrastive Learning](https://mlsanity.com/paper/2210.15265/) (2022)
- [Structured Dialogue Discourse Parsing](https://mlsanity.com/paper/2306.15103/) (2023)
- [Core-sets for Fair and Diverse Data Summarization](https://mlsanity.com/paper/2310.08122/) (2023)
- [Zero-Shot Dialogue Disentanglement by Self-Supervised Entangled Response Selection](https://mlsanity.com/paper/2110.12646/) (2021)
- [Dramatic Conversation Disentanglement](https://mlsanity.com/paper/2305.16648/) (2023)
- [Revisiting Conversation Discourse for Dialogue Disentanglement](https://mlsanity.com/paper/2306.03975/) (2023)
- [SEOVER: Sentence-level Emotion Orientation Vector based Conversation Emotion Recognition Model](https://mlsanity.com/paper/2106.08785/) (2021)
- [Learning Locality and Isotropy in Dialogue Modeling](https://mlsanity.com/paper/2205.14583/) (2022)
- [The Branch Not Taken: Predicting Branching in Online Conversations](https://mlsanity.com/paper/2404.13613/) (2024)
- [Disentangling Online Chats with DAG-Structured LSTMs](https://mlsanity.com/paper/2106.09024/) (2021)
- [Prompt-Based Learning for Thread Structure Prediction in Cybersecurity Forums](https://mlsanity.com/paper/2303.05400/) (2023)
- [Multi-party Response Generation with Relation Disentanglement](https://mlsanity.com/paper/2403.10827/) (2024)

## Abstract (arXiv)

Conversation structure is useful for both understanding the nature of conversation dynamics and for providing features for many downstream applications such as summarization of conversations. In this work, we define the problem of conversation structure modeling as identifying the parent utterance(s) to which each utterance in the conversation responds to. Previous work usually took a pair of utterances to decide whether one utterance is the parent of the other. We believe the entire ancestral history is a very important information source to make accurate prediction. Therefore, we design a novel masking mechanism to guide the ancestor flow, and leverage the transformer model to aggregate all ancestors to predict parent utterances. Our experiments are performed on the Reddit dataset (Zhang, Culbertson, and Paritosh 2017) and the Ubuntu IRC dataset (Kummerfeld et al. 2019). In addition, we also report experiments on a new larger corpus from the Reddit platform and release this dataset. We show that the proposed model, that takes into account the ancestral history of the conversation, significantly outperforms several strong baselines including the BERT model on all datasets