# Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detection

arXiv: https://arxiv.org/abs/2010.15360 · HTML: https://mlsanity.com/paper/2010.15360/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning](https://mlsanity.com/paper/2110.04429/) (2021)
- [Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Data](https://mlsanity.com/paper/2102.01226/) (2021)
- [Discriminative Self-training for Punctuation Prediction](https://mlsanity.com/paper/2104.10339/) (2021)
- [Span Classification with Structured Information for Disfluency Detection in Spoken Utterances](https://mlsanity.com/paper/2203.16028/) (2022)
- [Boosting Disfluency Detection with Large Language Model as Disfluency Generator](https://mlsanity.com/paper/2403.08229/) (2024)
- [CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2210.05146/) (2022)
- [A novel multimodal dynamic fusion network for disfluency detection in spoken utterances](https://mlsanity.com/paper/2211.14700/) (2022)
- [Z-Scores: A Metric for Linguistically Assessing Disfluency Removal](https://mlsanity.com/paper/2509.20319/) (2025)
- [Improving Distinction between ASR Errors and Speech Disfluencies with Feature Space Interpolation](https://mlsanity.com/paper/2108.01812/) (2021)
- [An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification](https://mlsanity.com/paper/2510.11537/) (2025)

## Abstract (arXiv)

Most existing approaches to disfluency detection heavily rely on human-annotated corpora, which is expensive to obtain in practice. There have been several proposals to alleviate this issue with, for instance, self-supervised learning techniques, but they still require human-annotated corpora. In this work, we explore the unsupervised learning paradigm which can potentially work with unlabeled text corpora that are cheaper and easier to obtain. Our model builds upon the recent work on Noisy Student Training, a semi-supervised learning approach that extends the idea of self-training. Experimental results on the commonly used English Switchboard test set show that our approach achieves competitive performance compared to the previous state-of-the-art supervised systems using contextualized word embeddings (e.g. BERT and ELECTRA).