# A Semi-supervised Learning Approach with Two Teachers to Improve Breakdown Identification in Dialogues

arXiv: https://arxiv.org/abs/2202.10948 · HTML: https://mlsanity.com/paper/2202.10948/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation.
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 22 Feb 2022.

## Placed in

- Knowledge distillation / [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/)

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [Training Deep Neural Networks on Noisy Labels with Bootstrapping](https://mlsanity.com/paper/1412.6596/) (2014)
- [Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning](https://mlsanity.com/paper/2012.09816/) (2020)
- [Adversarial Training Methods for Semi-Supervised Text Classification](https://mlsanity.com/paper/1605.07725/) (2016)
- [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020)
- [MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://mlsanity.com/paper/2004.12239/) (2020)
- [Self-Distillation Amplifies Regularization in Hilbert Space](https://mlsanity.com/paper/2002.05715/) (2020)
- [SSMBA: Self-Supervised Manifold Based Data Augmentation for Improving Out-of-Domain Robustness](https://mlsanity.com/paper/2009.10195/) (2020)
- [Learning to Self-Train for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1906.00562/) (2019)
- [Noisy Self-Knowledge Distillation for Text Summarization](https://mlsanity.com/paper/2009.07032/) (2020)
- [Adaptive Semi-supervised Learning for Cross-domain Sentiment Classification](https://mlsanity.com/paper/1809.00530/) (2018)
- [Unsupervised Domain Adaptation of Language Models for Reading Comprehension](https://mlsanity.com/paper/1911.10768/) (2019)
- [Improving Dialogue Breakdown Detection with Semi-Supervised Learning](https://mlsanity.com/paper/2011.00136/) (2020)
- [RSL19BD at DBDC4: Ensemble of Decision Tree-based and LSTM-based Models](https://mlsanity.com/paper/1905.01799/) (2019)

## Built on by (most cited first, 3 of 3)

- [Leveraging Large Language Models for Automated Dialogue Analysis](https://mlsanity.com/paper/2309.06490/) (2023)
- [Investigating Masking-based Data Generation in Language Models](https://mlsanity.com/paper/2307.00008/) (2023)
- [Detect, Explain, Escalate: Sustainable Dialogue Breakdown Management for LLM Agents](https://mlsanity.com/paper/2504.18839/) (2025)

## Abstract (arXiv)

Identifying breakdowns in ongoing dialogues helps to improve communication effectiveness. Most prior work on this topic relies on human annotated data and data augmentation to learn a classification model. While quality labeled dialogue data requires human annotation and is usually expensive to obtain, unlabeled data is easier to collect from various sources. In this paper, we propose a novel semi-supervised teacher-student learning framework to tackle this task. We introduce two teachers which are trained on labeled data and perturbed labeled data respectively. We leverage unlabeled data to improve classification in student training where we employ two teachers to refine the labeling of unlabeled data through teacher-student learning in a bootstrapping manner. Through our proposed training approach, the student can achieve improvements over single-teacher performance. Experimental results on the Dialogue Breakdown Detection Challenge dataset DBDC5 and Learning to Identify Follow-Up Questions dataset LIF show that our approach outperforms all previous published approaches as well as other supervised and semi-supervised baseline methods.