# Multitask-Based Joint Learning Approach To Robust ASR For Radio Communication Speech

arXiv: https://arxiv.org/abs/2107.10701 · HTML: https://mlsanity.com/paper/2107.10701/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2021.

## Built on by (most cited first, 11 of 11)

- [Noise-robust Speech Recognition with 10 Minutes Unparalleled In-domain Data](https://mlsanity.com/paper/2203.15321/) (2022)
- [Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition](https://mlsanity.com/paper/2110.05267/) (2021)
- [Gradient Remedy for Multi-Task Learning in End-to-End Noise-Robust Speech Recognition](https://mlsanity.com/paper/2302.11362/) (2023)
- [Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASR](https://mlsanity.com/paper/2304.04974/) (2023)
- [Self-critical Sequence Training for Automatic Speech Recognition](https://mlsanity.com/paper/2204.06260/) (2022)
- [Dual-Branch Knowledge Distillation for Noise-Robust Synthetic Speech Detection](https://mlsanity.com/paper/2310.08869/) (2023)
- [ROSE: A Recognition-Oriented Speech Enhancement Framework in Air Traffic Control Using Multi-Objective Learning](https://mlsanity.com/paper/2312.06118/) (2023)
- [Audio Enhancement for Computer Audition -- An Iterative Training Paradigm Using Sample Importance](https://mlsanity.com/paper/2408.06264/) (2024)
- [D4AM: A General Denoising Framework for Downstream Acoustic Models](https://mlsanity.com/paper/2311.16595/) (2023)
- [On the Efficacy and Noise-Robustness of Jointly Learned Speech Emotion and Automatic Speech Recognition](https://mlsanity.com/paper/2305.12540/) (2023)
- [Training-Free Intelligibility-Guided Observation Addition for Noisy ASR](https://mlsanity.com/paper/2602.20967/) (2026)

## Abstract (arXiv)

To realize robust end-to-end Automatic Speech Recognition(E2E ASR) under radio communication condition, we propose a multitask-based method to joint train a Speech Enhancement (SE) module as the front-end and an E2E ASR model as the back-end in this paper. One of the advantage of the proposed method is that the entire system can be trained from scratch. Different from prior works, either component here doesn't need to perform pre-training and fine-tuning processes separately. Through analysis, we found that the success of the proposed method lies in the following aspects. Firstly, multitask learning is essential, that is the SE network is not only learning to produce more Intelligent speech, it is also aimed to generate speech that is beneficial to recognition. Secondly, we also found speech phase preserved from noisy speech is critical for improving ASR performance. Thirdly, we propose a dual channel data augmentation training method to obtain further improvement.Specifically, we combine the clean and enhanced speech to train the whole system. We evaluate the proposed method on the RATS English data set, achieving a relative WER reduction of 4.6% with the joint training method, and up to a relative WER reduction of 11.2% with the proposed data augmentation method.