# Improving Distinction between ASR Errors and Speech Disfluencies with Feature Space Interpolation

arXiv: https://arxiv.org/abs/2108.01812 · HTML: https://mlsanity.com/paper/2108.01812/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 4 Aug 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [Improved Noisy Student Training for Automatic Speech Recognition](https://mlsanity.com/paper/2005.09629/) (2020)
- [Training Tips for the Transformer Model](https://mlsanity.com/paper/1804.00247/) (2018)
- [Sequence-Level Mixed Sample Data Augmentation](https://mlsanity.com/paper/2011.09039/) (2020)
- [Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Model](https://mlsanity.com/paper/1910.10697/) (2019)
- [SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup](https://mlsanity.com/paper/2010.02322/) (2020)
- [ASR Error Correction and Domain Adaptation Using Machine Translation](https://mlsanity.com/paper/2003.07692/) (2020)
- [Joint Contextual Modeling for ASR Correction and Language Understanding](https://mlsanity.com/paper/2002.00750/) (2020)
- [Multi-Task Self-Supervised Learning for Disfluency Detection](https://mlsanity.com/paper/1908.05378/) (2019)
- [Data Augmentation for Training Dialog Models Robust to Speech Recognition Errors](https://mlsanity.com/paper/2006.05635/) (2020)
- [Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data](https://mlsanity.com/paper/2010.11506/) (2020)
- [Semi-Supervised Speech Recognition via Local Prior Matching](https://mlsanity.com/paper/2002.10336/) (2020)
- [Combining Self-Training and Self-Supervised Learning for Unsupervised Disfluency Detection](https://mlsanity.com/paper/2010.15360/) (2020)
- [Towards Fluent Translations from Disfluent Speech](https://mlsanity.com/paper/1811.03189/) (2018)
- [Auxiliary Sequence Labeling Tasks for Disfluency Detection](https://mlsanity.com/paper/2011.04512/) (2020)
- [Adapting general-purpose speech recognition engine output for domain-specific natural language question answering](https://mlsanity.com/paper/1710.06923/) (2017)

## Abstract (arXiv)

Fine-tuning pretrained language models (LMs) is a popular approach to automatic speech recognition (ASR) error detection during post-processing. While error detection systems often take advantage of statistical language archetypes captured by LMs, at times the pretrained knowledge can hinder error detection performance. For instance, presence of speech disfluencies might confuse the post-processing system into tagging disfluent but accurate transcriptions as ASR errors. Such confusion occurs because both error detection and disfluency detection tasks attempt to identify tokens at statistically unlikely positions. This paper proposes a scheme to improve existing LM-based ASR error detection systems, both in terms of detection scores and resilience to such distracting auxiliary tasks. Our approach adopts the popular mixup method in text feature space and can be utilized with any black-box ASR output. To demonstrate the effectiveness of our method, we conduct post-processing experiments with both traditional and end-to-end ASR systems (both for English and Korean languages) with 5 different speech corpora. We find that our method improves both ASR error detection F 1 scores and reduces the number of correctly transcribed disfluencies wrongly detected as ASR errors. Finally, we suggest methods to utilize resulting LMs directly in semi-supervised ASR training.