# Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots

arXiv: https://arxiv.org/abs/2004.03588 · HTML: https://mlsanity.com/paper/2004.03588/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 30 of 73)

- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding](https://mlsanity.com/paper/2106.01541/) (2021)
- [LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically Constructed from Live Streaming](https://mlsanity.com/paper/2306.08401/) (2023)
- [Graph Based Network with Contextualized Representations of Turns in Dialogue](https://mlsanity.com/paper/2109.04008/) (2021)
- [CPED: A Large-Scale Chinese Personalized and Emotional Dialogue Dataset for Conversational AI](https://mlsanity.com/paper/2205.14727/) (2022)
- [On the Calibration and Uncertainty of Neural Learning to Rank Models](https://mlsanity.com/paper/2101.04356/) (2021)
- [Structural Pre-training for Dialogue Comprehension](https://mlsanity.com/paper/2105.10956/) (2021)
- [Agent4Ranking: Semantic Robust Ranking via Personalized Query Rewriting Using Multi-agent LLM](https://mlsanity.com/paper/2312.15450/) (2023)
- [Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation](https://mlsanity.com/paper/2106.05894/) (2021)
- [HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations](https://mlsanity.com/paper/2203.08500/) (2022)
- [Partner Matters! An Empirical Study on Fusing Personas for Personalized Response Selection in Retrieval-Based Chatbots](https://mlsanity.com/paper/2105.09050/) (2021)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [Self- and Pseudo-self-supervised Prediction of Speaker and Key-utterance for Multi-party Dialogue Reading Comprehension](https://mlsanity.com/paper/2109.03772/) (2021)
- [Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehension](https://mlsanity.com/paper/2109.04066/) (2021)
- [Advances in Multi-turn Dialogue Comprehension: A Survey](https://mlsanity.com/paper/2103.03125/) (2021)
- [Exploring Dense Retrieval for Dialogue Response Selection](https://mlsanity.com/paper/2110.06612/) (2021)
- [SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation](https://mlsanity.com/paper/2405.15924/) (2024)
- [GIFT: Graph-Induced Fine-Tuning for Multi-Party Conversation Understanding](https://mlsanity.com/paper/2305.09360/) (2023)
- [Dialogue-oriented Pre-training](https://mlsanity.com/paper/2106.00420/) (2021)
- [Integrating Dialog History into End-to-End Spoken Language Understanding Systems](https://mlsanity.com/paper/2108.08405/) (2021)
- [Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations](https://mlsanity.com/paper/2109.13087/) (2021)
- [P5: Plug-and-Play Persona Prompting for Personalized Response Selection](https://mlsanity.com/paper/2310.06390/) (2023)
- [Structural Characterization for Dialogue Disentanglement](https://mlsanity.com/paper/2110.08018/) (2021)
- [Multi-Stage Pre-training Enhanced by ChatGPT for Multi-Scenario Multi-Domain Dialogue Summarization](https://mlsanity.com/paper/2310.10285/) (2023)
- [Multi-Party Conversational Agents: A Survey](https://mlsanity.com/paper/2505.18845/) (2025)
- [Persona-Based Conversational AI: State of the Art and Challenges](https://mlsanity.com/paper/2212.03699/) (2022)
- [Hierarchical Dialogue Understanding with Special Tokens and Turn-level Attention](https://mlsanity.com/paper/2305.00262/) (2023)
- [MADNet: Maximizing Addressee Deduction Expectation for Multi-Party Conversation Generation](https://mlsanity.com/paper/2305.12733/) (2023)
- [Improving Response Quality with Backward Reasoning in Open-domain Dialogue Systems](https://mlsanity.com/paper/2105.00079/) (2021)
- [Small Changes Make Big Differences: Improving Multi-turn Response Selection in Dialogue Systems via Fine-Grained Contrastive Learning](https://mlsanity.com/paper/2111.10154/) (2021)

## Abstract (arXiv)

In this paper, we study the problem of employing pre-trained language models for multi-turn response selection in retrieval-based chatbots. A new model, named Speaker-Aware BERT (SA-BERT), is proposed in order to make the model aware of the speaker change information, which is an important and intrinsic property of multi-turn dialogues. Furthermore, a speaker-aware disentanglement strategy is proposed to tackle the entangled dialogues. This strategy selects a small number of most important utterances as the filtered context according to the speakers' information in them. Finally, domain adaptation is performed to incorporate the in-domain knowledge into pre-trained language models. Experiments on five public datasets show that our proposed model outperforms the present models on all metrics by large margins and achieves new state-of-the-art performances for multi-turn response selection.