# DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement

arXiv: https://arxiv.org/abs/2004.03760 · HTML: https://mlsanity.com/paper/2004.03760/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2020.

## Built on by (most cited first, 12 of 12)

- [Structural Characterization for Dialogue Disentanglement](https://mlsanity.com/paper/2110.08018/) (2021)
- [Multi-Party Conversational Agents: A Survey](https://mlsanity.com/paper/2505.18845/) (2025)
- [Conversation Disentanglement with Bi-Level Contrastive Learning](https://mlsanity.com/paper/2210.15265/) (2022)
- [A Review and Roadmap of Deep Learning Causal Discovery in Different Variable Paradigms](https://mlsanity.com/paper/2209.06367/) (2022)
- [Structured Dialogue Discourse Parsing](https://mlsanity.com/paper/2306.15103/) (2023)
- [Zero-Shot Dialogue Disentanglement by Self-Supervised Entangled Response Selection](https://mlsanity.com/paper/2110.12646/) (2021)
- [Revisiting Conversation Discourse for Dialogue Disentanglement](https://mlsanity.com/paper/2306.03975/) (2023)
- [Thread Detection and Response Generation using Transformers with Prompt Optimisation](https://mlsanity.com/paper/2403.05931/) (2024)
- [Disentangling Online Chats with DAG-Structured LSTMs](https://mlsanity.com/paper/2106.09024/) (2021)
- [Pre-trained Language Models as Re-Annotators](https://mlsanity.com/paper/2205.05368/) (2022)
- [Local and Global Contexts for Conversation](https://mlsanity.com/paper/2401.17588/) (2024)
- [Disentangling Genotype and Environment Specific Latent Features for Improved Trait Prediction using a Compositional Autoencoder](https://mlsanity.com/paper/2410.19922/) (2024)

## Abstract (arXiv)

Disentanglement is a problem in which multiple conversations occur in the same channel simultaneously, and the listener should decide which utterance is part of the conversation he will respond to. We propose a new model, named Dialogue BERT (DialBERT), which integrates local and global semantics in a single stream of messages to disentangle the conversations that mixed together. We employ BERT to capture the matching information in each utterance pair at the utterance-level, and use a BiLSTM to aggregate and incorporate the context-level information. With only a 3% increase in parameters, a 12% improvement has been attained in comparison to BERT, based on the F1-Score. The model achieves a state-of-the-art result on the a new dataset proposed by IBM and surpasses previous work by a substantial margin.