# Topic Shift Detection in Chinese Dialogues: Corpus and Benchmark

arXiv: https://arxiv.org/abs/2305.01195 · HTML: https://mlsanity.com/paper/2305.01195/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 27 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset](https://mlsanity.com/paper/1710.03957/) (2017)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [Key-Value Retrieval Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1705.05414/) (2017)
- [Proactive Human-Machine Conversation with Explicit Conversation Goals](https://mlsanity.com/paper/1906.05572/) (2019)
- [Text Segmentation as a Supervised Learning Task](https://mlsanity.com/paper/1803.09337/) (2018)
- [Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1804.06512/) (2018)
- [SECTOR: A Neural Model for Coherent Topic Segmentation and Classification](https://mlsanity.com/paper/1902.04793/) (2019)
- [Topic-Aware Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2009.12539/) (2020)
- [Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems](https://mlsanity.com/paper/1906.09308/) (2019)
- [NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversation](https://mlsanity.com/paper/2103.02548/) (2021)
- [Contrast and Generation Make BART a Good Dialogue Emotion Recognizer](https://mlsanity.com/paper/2112.11202/) (2021)
- [Hint-Based Training for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/1909.06708/) (2019)
- [Attention-based Neural Text Segmentation](https://mlsanity.com/paper/1808.09935/) (2018)
- [TIAGE: A Benchmark for Topic-Shift Aware Dialog Modeling](https://mlsanity.com/paper/2109.04562/) (2021)
- [Regularizing Dialogue Generation by Imitating Implicit Scenarios](https://mlsanity.com/paper/2010.01893/) (2020)
- [DADgraph: A Discourse-aware Dialogue Graph Neural Network for Multiparty Dialogue Machine Reading Comprehension](https://mlsanity.com/paper/2104.12377/) (2021)
- [Improving Robustness of Task Oriented Dialog Systems](https://mlsanity.com/paper/1911.05153/) (2019)
- [Self- and Pseudo-self-supervised Prediction of Speaker and Key-utterance for Multi-party Dialogue Reading Comprehension](https://mlsanity.com/paper/2109.03772/) (2021)

## Built on by (most cited first, 4 of 4)

- [Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study](https://mlsanity.com/paper/2305.08391/) (2023)
- [DiaHalu: A Dialogue-level Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2403.00896/) (2024)
- [Multi-Granularity Prompts for Topic Shift Detection in Dialogue](https://mlsanity.com/paper/2305.14006/) (2023)
- [Unsupervised Mutual Learning of Discourse Parsing and Topic Segmentation in Dialogue](https://mlsanity.com/paper/2405.19799/) (2024)

## Abstract (arXiv)

Dialogue topic shift detection is to detect whether an ongoing topic has shifted or should shift in a dialogue, which can be divided into two categories, i.e., response-known task and response-unknown task. Currently, only a few investigated the latter, because it is still a challenge to predict the topic shift without the response information. In this paper, we first annotate a Chinese Natural Topic Dialogue (CNTD) corpus consisting of 1308 dialogues to fill the gap in the Chinese natural conversation topic corpus. And then we focus on the response-unknown task and propose a teacher-student framework based on hierarchical contrastive learning to predict the topic shift without the response. Specifically, the response at high-level teacher-student is introduced to build the contrastive learning between the response and the context, while the label contrastive learning is constructed at low-level student. The experimental results on our Chinese CNTD and English TIAGE show the effectiveness of our proposed model.