# Multi-Task Self-Supervised Learning for Disfluency Detection

arXiv: https://arxiv.org/abs/1908.05378 · HTML: https://mlsanity.com/paper/1908.05378/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2019.

## Built on by (most cited first, 20 of 20)

- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answering](https://mlsanity.com/paper/2106.04016/) (2021)
- [TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations](https://mlsanity.com/paper/2112.12441/) (2021)
- [RPT: Toward Transferable Model on Heterogeneous Researcher Data via Pre-Training](https://mlsanity.com/paper/2110.07336/) (2021)
- [Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics](https://mlsanity.com/paper/2112.08321/) (2021)
- [LARD: Large-scale Artificial Disfluency Generation](https://mlsanity.com/paper/2201.05041/) (2022)
- [Filler Word Detection and Classification: A Dataset and Benchmark](https://mlsanity.com/paper/2203.15135/) (2022)
- [Cross-Modal ASR Post-Processing System for Error Correction and Utterance Rejection](https://mlsanity.com/paper/2201.03313/) (2022)
- [Multi-Task Self-Supervised Time-Series Representation Learning](https://mlsanity.com/paper/2303.01034/) (2023)
- [Automatic Disfluency Detection from Untranscribed Speech](https://mlsanity.com/paper/2311.00867/) (2023)
- [What BERT Based Language Models Learn in Spoken Transcripts: An Empirical Study](https://mlsanity.com/paper/2109.09105/) (2021)
- [Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer](https://mlsanity.com/paper/2110.02950/) (2021)
- [Sound and Visual Representation Learning with Multiple Pretraining Tasks](https://mlsanity.com/paper/2201.01046/) (2022)
- [Transcription free filler word detection with Neural semi-CRFs](https://mlsanity.com/paper/2303.06475/) (2023)
- [Boosting Disfluency Detection with Large Language Model as Disfluency Generator](https://mlsanity.com/paper/2403.08229/) (2024)
- [Contextualizing Variation in Text Style Transfer Datasets](https://mlsanity.com/paper/2108.07871/) (2021)
- [Artificial Disfluency Detection, Uh No, Disfluency Generation for the Masses](https://mlsanity.com/paper/2211.09235/) (2022)
- [Improving Distinction between ASR Errors and Speech Disfluencies with Feature Space Interpolation](https://mlsanity.com/paper/2108.01812/) (2021)
- [End-to-End Speech Recognition and Disfluency Removal with Acoustic Language Model Pretraining](https://mlsanity.com/paper/2309.04516/) (2023)
- [Looks can be Deceptive: Distinguishing Repetition Disfluency from Reduplication](https://mlsanity.com/paper/2407.08147/) (2024)

## Abstract (arXiv)

Most existing approaches to disfluency detection heavily rely on human-annotated data, which is expensive to obtain in practice. To tackle the training data bottleneck, we investigate methods for combining multiple self-supervised tasks-i.e., supervised tasks where data can be collected without manual labeling. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled news data, and propose two self-supervised pre-training tasks: (i) tagging task to detect the added noisy words. (ii) sentence classification to distinguish original sentences from grammatically-incorrect sentences. We then combine these two tasks to jointly train a network. The pre-trained network is then fine-tuned using human-annotated disfluency detection training data. Experimental results on the commonly used English Switchboard test set show that our approach can achieve competitive performance compared to the previous systems (trained using the full dataset) by using less than 1% (1000 sentences) of the training data. Our method trained on the full dataset significantly outperforms previous methods, reducing the error by 21% on English Switchboard.