# Self-Supervised Dialogue Learning

arXiv: https://arxiv.org/abs/1907.00448 · HTML: https://mlsanity.com/paper/1907.00448/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2019.

## Built on by (most cited first, 16 of 16)

- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Self-Supervised Learning of Graph Neural Networks: A Unified Review](https://mlsanity.com/paper/2102.10757/) (2021)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Crystal Twins: Self-supervised Learning for Crystalline Material Property Prediction](https://mlsanity.com/paper/2205.01893/) (2022)
- [Self-supervised Dialogue Learning for Spoken Conversational Question Answering](https://mlsanity.com/paper/2106.02182/) (2021)
- [Self-Supervised Representation Learning via Latent Graph Prediction](https://mlsanity.com/paper/2202.08333/) (2022)
- [Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders](https://mlsanity.com/paper/2106.03410/) (2021)
- [Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision](https://mlsanity.com/paper/2107.08582/) (2021)
- [Distribution Estimation to Automate Transformation Policies for Self-Supervision](https://mlsanity.com/paper/2111.12265/) (2021)
- [The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues](https://mlsanity.com/paper/2103.11151/) (2021)
- [WeaSuL: Weakly Supervised Dialogue Policy Learning: Reward Estimation for Multi-turn Dialogue](https://mlsanity.com/paper/2108.01487/) (2021)
- [Re-entry Prediction for Online Conversations via Self-Supervised Learning](https://mlsanity.com/paper/2109.02020/) (2021)
- [Enhancing Semantic Understanding with Self-supervised Methods for Abstractive Dialogue Summarization](https://mlsanity.com/paper/2209.00278/) (2022)
- [CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2210.05146/) (2022)
- [UserBERT: Modeling Long- and Short-Term User Preferences via Self-Supervision](https://mlsanity.com/paper/2202.07605/) (2022)
- [Towards Universal Large-Scale Foundational Model for Natural Gas Demand Forecasting](https://mlsanity.com/paper/2409.15794/) (2024)

## Abstract (arXiv)

The sequential order of utterances is often meaningful in coherent dialogues, and the order changes of utterances could lead to low-quality and incoherent conversations. We consider the order information as a crucial supervised signal for dialogue learning, which, however, has been neglected by many previous dialogue systems. Therefore, in this paper, we introduce a self-supervised learning task, inconsistent order detection, to explicitly capture the flow of conversation in dialogues. Given a sampled utterance pair triple, the task is to predict whether it is ordered or misordered. Then we propose a sampling-based self-supervised network SSN to perform the prediction with sampled triple references from previous dialogue history. Furthermore, we design a joint learning framework where SSN can guide the dialogue systems towards more coherent and relevant dialogue learning through adversarial training. We demonstrate that the proposed methods can be applied to both open-domain and task-oriented dialogue scenarios, and achieve the new state-of-the-art performance on the OpenSubtitiles and Movie-Ticket Booking datasets.