# Improving Disfluency Detection by Self-Training a Self-Attentive Model

arXiv: https://arxiv.org/abs/2004.05323 · HTML: https://mlsanity.com/paper/2004.05323/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2020.

## Built on by (most cited first, 25 of 25)

- [Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answering](https://mlsanity.com/paper/2106.04016/) (2021)
- [Disfluency Detection with Unlabeled Data and Small BERT Models](https://mlsanity.com/paper/2104.10769/) (2021)
- [Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment](https://mlsanity.com/paper/2305.12260/) (2023)
- [Span Classification with Structured Information for Disfluency Detection in Spoken Utterances](https://mlsanity.com/paper/2203.16028/) (2022)
- [Four-in-One: A Joint Approach to Inverse Text Normalization, Punctuation, Capitalization, and Disfluency for Automatic Speech Recognition](https://mlsanity.com/paper/2210.15063/) (2022)
- [Fillers in Spoken Language Understanding: Computational and Psycholinguistic Perspectives](https://mlsanity.com/paper/2301.10761/) (2023)
- [Adversarial Training For Low-Resource Disfluency Correction](https://mlsanity.com/paper/2306.06384/) (2023)
- [From Disfluency Detection to Intent Detection and Slot Filling](https://mlsanity.com/paper/2209.08359/) (2022)
- [MultiTurnCleanup: A Benchmark for Multi-Turn Spoken Conversational Transcript Cleanup](https://mlsanity.com/paper/2305.12029/) (2023)
- [VAKTA-SETU: A Speech-to-Speech Machine Translation Service in Select Indic Languages](https://mlsanity.com/paper/2305.12518/) (2023)
- [Boosting Disfluency Detection with Large Language Model as Disfluency Generator](https://mlsanity.com/paper/2403.08229/) (2024)
- [Conversational Speech Reveals Structural Robustness Failures in SpeechLLM Backbones](https://mlsanity.com/paper/2509.20321/) (2025)
- [Artificial Disfluency Detection, Uh No, Disfluency Generation for the Masses](https://mlsanity.com/paper/2211.09235/) (2022)
- [A novel multimodal dynamic fusion network for disfluency detection in spoken utterances](https://mlsanity.com/paper/2211.14700/) (2022)
- [Structured Tree Alignment for Evaluation of (Speech) Constituency Parsing](https://mlsanity.com/paper/2402.13433/) (2024)
- [Humane Speech Synthesis through Zero-Shot Emotion and Disfluency Generation](https://mlsanity.com/paper/2404.01339/) (2024)
- [Z-Scores: A Metric for Linguistically Assessing Disfluency Removal](https://mlsanity.com/paper/2509.20319/) (2025)
- [FormalASR: End-to-End Spoken Chinese to Formal Text](https://mlsanity.com/paper/2605.19266/) (2026)
- [Improved POS tagging for spontaneous, clinical speech using data augmentation](https://mlsanity.com/paper/2307.05796/) (2023)
- [Adapting the NICT-JLE Corpus for Disfluency Detection Models](https://mlsanity.com/paper/2308.02482/) (2023)
- [End-to-End Speech Recognition and Disfluency Removal with Acoustic Language Model Pretraining](https://mlsanity.com/paper/2309.04516/) (2023)
- [Speakerly: A Voice-based Writing Assistant for Text Composition](https://mlsanity.com/paper/2310.16251/) (2023)
- [Looks can be Deceptive: Distinguishing Repetition Disfluency from Reduplication](https://mlsanity.com/paper/2407.08147/) (2024)
- [Masculine Defaults via Gendered Discourse in Podcasts and Large Language Models](https://mlsanity.com/paper/2504.11431/) (2025)
- [An Encoder-Integrated PhoBERT with Graph Attention for Vietnamese Token-Level Classification](https://mlsanity.com/paper/2510.11537/) (2025)

## Abstract (arXiv)

Self-attentive neural syntactic parsers using contextualized word embeddings (e.g. ELMo or BERT) currently produce state-of-the-art results in joint parsing and disfluency detection in speech transcripts. Since the contextualized word embeddings are pre-trained on a large amount of unlabeled data, using additional unlabeled data to train a neural model might seem redundant. However, we show that self-training - a semi-supervised technique for incorporating unlabeled data - sets a new state-of-the-art for the self-attentive parser on disfluency detection, demonstrating that self-training provides benefits orthogonal to the pre-trained contextualized word representations. We also show that ensembling self-trained parsers provides further gains for disfluency detection.