# Data Augmentation for Training Dialog Models Robust to Speech Recognition Errors

arXiv: https://arxiv.org/abs/2006.05635 · HTML: https://mlsanity.com/paper/2006.05635/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 17 of 17)

- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- ["How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations](https://mlsanity.com/paper/2109.13489/) (2021)
- ["Do you follow me?": A Survey of Recent Approaches in Dialogue State Tracking](https://mlsanity.com/paper/2207.14627/) (2022)
- [Revisiting the Boundary between ASR and NLU in the Age of Conversational Dialog Systems](https://mlsanity.com/paper/2112.05842/) (2021)
- [TOD-DA: Towards Boosting the Robustness of Task-oriented Dialogue Modeling on Spoken Conversations](https://mlsanity.com/paper/2112.12441/) (2021)
- [ASR-GLUE: A New Multi-task Benchmark for ASR-Robust Natural Language Understanding](https://mlsanity.com/paper/2108.13048/) (2021)
- [An Approach to Improve Robustness of NLP Systems against ASR Errors](https://mlsanity.com/paper/2103.13610/) (2021)
- [MEDSAGE: Enhancing Robustness of Medical Dialogue Summarization to ASR Errors with LLM-generated Synthetic Dialogues](https://mlsanity.com/paper/2408.14418/) (2024)
- [OLISIA: a Cascade System for Spoken Dialogue State Tracking](https://mlsanity.com/paper/2304.11073/) (2023)
- [Toward Practical Automatic Speech Recognition and Post-Processing: a Call for Explainable Error Benchmark Guideline](https://mlsanity.com/paper/2401.14625/) (2024)
- [Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learning](https://mlsanity.com/paper/2209.07873/) (2022)
- [Enhancing Dialogue Speech Recognition with Robust Contextual Awareness via Noise Representation Learning](https://mlsanity.com/paper/2408.06043/) (2024)
- [Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition](https://mlsanity.com/paper/2401.10447/) (2024)
- [Interventional Speech Noise Injection for ASR Generalizable Spoken Language Understanding](https://mlsanity.com/paper/2410.15609/) (2024)
- [Improving Distinction between ASR Errors and Speech Disfluencies with Feature Space Interpolation](https://mlsanity.com/paper/2108.01812/) (2021)
- [Exploring the Usage of Chinese Pinyin in Pretraining](https://mlsanity.com/paper/2310.04960/) (2023)
- [Curriculum-Based Noise Adaptation for Phoneme-to-Text Reconstruction in Visual Speech Recognition](https://mlsanity.com/paper/2609.20839/) (2026)

## Abstract (arXiv)

Speech-based virtual assistants, such as Amazon Alexa, Google assistant, and Apple Siri, typically convert users' audio signals to text data through automatic speech recognition (ASR) and feed the text to downstream dialog models for natural language understanding and response generation. The ASR output is error-prone; however, the downstream dialog models are often trained on error-free text data, making them sensitive to ASR errors during inference time. To bridge the gap and make dialog models more robust to ASR errors, we leverage an ASR error simulator to inject noise into the error-free text data, and subsequently train the dialog models with the augmented data. Compared to other approaches for handling ASR errors, such as using ASR lattice or end-to-end methods, our data augmentation approach does not require any modification to the ASR or downstream dialog models; our approach also does not introduce any additional latency during inference time. We perform extensive experiments on benchmark data and show that our approach improves the performance of downstream dialog models in the presence of ASR errors, and it is particularly effective in the low-resource situations where there are constraints on model size or the training data is scarce.