# Dual Application of Speech Enhancement for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2011.03840 · HTML: https://mlsanity.com/paper/2011.03840/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [Large Language Models are Efficient Learners of Noise-Robust Speech Recognition](https://mlsanity.com/paper/2401.10446/) (2024)
- [Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition](https://mlsanity.com/paper/2110.05267/) (2021)
- [Dual-Path Style Learning for End-to-End Noise-Robust Speech Recognition](https://mlsanity.com/paper/2203.14838/) (2022)
- [Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation](https://mlsanity.com/paper/2302.11131/) (2023)
- [Gradient Remedy for Multi-Task Learning in End-to-End Noise-Robust Speech Recognition](https://mlsanity.com/paper/2302.11362/) (2023)
- [Dual-Branch Knowledge Distillation for Noise-Robust Synthetic Speech Detection](https://mlsanity.com/paper/2310.08869/) (2023)
- [ROSE: A Recognition-Oriented Speech Enhancement Framework in Air Traffic Control Using Multi-Objective Learning](https://mlsanity.com/paper/2312.06118/) (2023)
- [Task-aware Warping Factors in Mask-based Speech Enhancement](https://mlsanity.com/paper/2108.12128/) (2021)
- [EMGSE: Acoustic/EMG Fusion for Multimodal Speech Enhancement](https://mlsanity.com/paper/2202.06507/) (2022)

## Abstract (arXiv)

In this work, we exploit speech enhancement for improving a recurrent neural network transducer (RNN-T) based ASR system. We employ a dense convolutional recurrent network (DCRN) for complex spectral mapping based speech enhancement, and find it helpful for ASR in two ways: a data augmentation technique, and a preprocessing frontend. In using it for ASR data augmentation, we exploit a KL divergence based consistency loss that is computed between the ASR outputs of original and enhanced utterances. In using speech enhancement as an effective ASR frontend, we propose a three-step training scheme based on model pretraining and feature selection. We evaluate our proposed techniques on a challenging social media English video dataset, and achieve an average relative improvement of 11.2% with speech enhancement based data augmentation, 8.3% with enhancement based preprocessing, and 13.4% when combining both.