speech and noise dual-stream spectrogram refine network with speech distortion loss for robust speech recognition
On the map: Speech and audio language models → Speech enhancement and noise suppression
Abstract (arXiv)
In recent years, the joint training of speech enhancement front-end and automatic speech recognition (ASR) back-end has been widely used to improve the robustness of ASR systems. Traditional joint training methods only use enhanced speech as input for the backend. However, it is difficult for speech enhancement systems to directly separate speech from input due to the diverse types of noise with different intensities. Furthermore, speech distortion and residual noise are often observed in enhanced speech, and the distortion of speech and noise is different. Most existing methods focus on fusing enhanced and noisy features to address this issue. In this paper, we propose a dual-stream spectrogram refine network to simultaneously refine the speech and noise and decouple the noise from the noisy input. Our proposed method can achieve better performance with a relative 8.6% CER reduction.
Key facts. Cited by 1 paper in the map (data through 19 Oct 2026). Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. Builds on 16 of its 23 references that are in the map. First posted on arXiv 29 May 2023.
Builds on
16 of 23 references.
Closest ancestors · 7
Same direction, newest first
- 2022Mask scalar prediction for improving robust automatic speech recognition26 Apr
- 2022Noise-robust Speech Recognition with 10 Minutes Unparalleled In-domain Data29 Mar
- 2022How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR18 Jan
- 2021Effect of noise suppression losses on speech distortion and ASR performance23 Nov
- 2021Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition11 Oct
- 2021Multitask-Based Joint Learning Approach To Robust ASR For Radio Communication Speech22 Jul
- 2021An Investigation of End-to-End Models for Robust Speech Recognition11 Feb
Other references · 9
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,636
- 2018ESPnet: End-to-End Speech Processing Toolkitcited by 754
- 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 497
- 2020VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognitioncited by 54
- 2021End-to-End Dereverberation, Beamforming, and Speech Recognition with Improved Numerical Stability and Advanced Frontendcited by 18
- 2020Gated Recurrent Fusion with Joint Training Framework for Robust End-to-End Speech Recognitioncited by 17
- 2019End-to-end training of time domain audio separation and recognitioncited by 13
- 2020Improved Robustness to Disfluencies in RNN-Transducer Based Speech Recognitioncited by 6
- 2022pMCT: Patched Multi-Condition Training for Robust Speech Recognitioncited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026DualSpecSE: A Dual-Path Speech Enhancement Network Integrating Mel and Complex Spectrograms
- 2024Mel-FullSubNet: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR
- 2025CleanMel: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR
- 2025Lightweight Front-end Enhancement for Robust ASR via Frame Resampling and Sub-Band Pruning
- 2022Joint Training of Speech Enhancement and Self-supervised Model for Noise-robust ASR
- 2026Latent-Level Enhancement with Flow Matching for Robust Automatic Speech Recognition
- 2022Dual-Path Style Learning for End-to-End Noise-Robust Speech Recognition
- 2023Bring the Noise: Introducing Noise Robustness to Pretrained Automatic Speech Recognition