# Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detection

arXiv: https://arxiv.org/abs/2205.00620 · HTML: https://mlsanity.com/paper/2205.00620/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 2 May 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Disfluency Detection using a Bidirectional LSTM](https://mlsanity.com/paper/1604.03209/) (2016)
- [Controllable Time-Delay Transformer for Real-Time Punctuation Prediction and Disfluency Detection](https://mlsanity.com/paper/2003.01309/) (2020)
- [Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answering](https://mlsanity.com/paper/2106.04016/) (2021)
- [Disfluency Detection with Unlabeled Data and Small BERT Models](https://mlsanity.com/paper/2104.10769/) (2021)
- [Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU](https://mlsanity.com/paper/2109.07364/) (2021)
- [End-to-End Speech Recognition and Disfluency Removal](https://mlsanity.com/paper/2009.10298/) (2020)
- [Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU](https://mlsanity.com/paper/2010.05330/) (2020)
- [Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena](https://mlsanity.com/paper/1709.07840/) (2017)
- [Re-framing Incremental Deep Language Models for Dialogue Processing with Multi-task Learning](https://mlsanity.com/paper/2011.06754/) (2020)
- [Multi-Task Learning for Domain-General Spoken Disfluency Detection in Dialogue Systems](https://mlsanity.com/paper/1810.03352/) (2018)

## Built on by (most cited first, 12 of 12)

- [Streaming Joint Speech Recognition and Disfluency Detection](https://mlsanity.com/paper/2211.08726/) (2022)
- [Augmenting Automatic Speech Recognition Models with Disfluency Detection](https://mlsanity.com/paper/2409.10177/) (2024)
- [Fillers in Spoken Language Understanding: Computational and Psycholinguistic Perspectives](https://mlsanity.com/paper/2301.10761/) (2023)
- [TAPIR: Learning Adaptive Revision for Incremental Natural Language Understanding with a Two-Pass Model](https://mlsanity.com/paper/2305.10845/) (2023)
- [MultiTurnCleanup: A Benchmark for Multi-Turn Spoken Conversational Transcript Cleanup](https://mlsanity.com/paper/2305.12029/) (2023)
- [The Road to Quality is Paved with Good Revisions: A Detailed Evaluation Methodology for Revision Policies in Incremental Sequence Labelling](https://mlsanity.com/paper/2307.15508/) (2023)
- [Artificial Disfluency Detection, Uh No, Disfluency Generation for the Masses](https://mlsanity.com/paper/2211.09235/) (2022)
- [DISCO: A Large Scale Human Annotated Corpus for Disfluency Correction in Indo-European Languages](https://mlsanity.com/paper/2310.16749/) (2023)
- [When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality](https://mlsanity.com/paper/2402.13113/) (2024)
- [5IDER: Unified Query Rewriting for Steering, Intent Carryover, Disfluencies, Entity Carryover and Repair](https://mlsanity.com/paper/2306.01855/) (2023)
- [Investigating large language models for their competence in extracting grammatically sound sentences from transcribed noisy utterances](https://mlsanity.com/paper/2410.05099/) (2024)
- [On the Difficulty of Token-Level Modeling of Dysfluency and Fluency Shaping Artifacts](https://mlsanity.com/paper/2512.02027/) (2025)

## Abstract (arXiv)

In modern interactive speech-based systems, speech is consumed and transcribed incrementally prior to having disfluencies removed. This post-processing step is crucial for producing clean transcripts and high performance on downstream tasks (e.g. machine translation). However, most current state-of-the-art NLP models such as the Transformer operate non-incrementally, potentially causing unacceptable delays. We propose a streaming BERT-based sequence tagging model that, combined with a novel training objective, is capable of detecting disfluencies in real-time while balancing accuracy and latency. This is accomplished by training the model to decide whether to immediately output a prediction for the current input or to wait for further context. Essentially, the model learns to dynamically size its lookahead window. Our results demonstrate that our model produces comparably accurate predictions and does so sooner than our baselines, with lower flicker. Furthermore, the model attains state-of-the-art latency and stability scores when compared with recent work on incremental disfluency detection.