Feature Learning and Ensemble Pre-Tasks Based Self-Supervised Speech Denoising and Dereverberation
On the map: Speech and audio language models → Speech enhancement and noise suppression
Abstract (arXiv)
Self-supervised learning (SSL) achieves great success in monaural speech enhancement, while the accuracy of the target speech estimation, particularly for unseen speakers, remains inadequate with existing pre-tasks. As speech signal contains multi-faceted information including speaker identity, paralinguistics, and spoken content, the latent representation for speech enhancement becomes a tough task. In this paper, we study the effectiveness of each feature which is commonly used in speech enhancement and exploit the feature combination in the SSL case. Besides, we propose an ensemble training strategy. The latent representation of the clean speech signal is learned, meanwhile, the dereverberated mask and the estimated ratio mask are exploited to denoise and dereverberate the mixture. The latent representation learning and the masks estimation are considered as two pre-tasks in the training stage. In addition, to study the effectiveness between the pre-tasks, we compare different training routines to train the model and further refine the performance. The NOISEX and DAPS corpora are used to evaluate the efficacy of the proposed method, which also outperforms the state-of-the-art methods.
Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. Builds on 17 of its 47 references that are in the map. First posted on arXiv 10 Jun 2022.
Builds on
17 of 47 references.
Closest ancestors · 1
Same direction, newest first
Other references · 16
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,093
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,762
- 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 342
- 2016Data Programming: Creating Large Training Sets, Quicklycited by 248
- 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 219
- 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 211
- 2020Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separationcited by 105
- 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 103
- 2020Self-supervised Learning for Speech Enhancementcited by 18
- 2017Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognitioncited by 11
- 2020Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-trainingcited by 10
4 more
- 2020Self-Supervised Learning from Contrastive Mixtures for Personalized Speech Enhancementcited by 7
- 2021Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuningcited by 6
- 2020Complex ratio masking for singing voice separationcited by 3
- 2021Improving Distantly Supervised Relation Extraction with Self-Ensemble Noise Filteringcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021Self-Supervised Learning based Monaural Speech Enhancement with Multi-Task Pre-Training
- 2025Generic Speech Enhancement with Self-Supervised Representation Space Loss
- 2023Feature Normalization for Fine-tuning Self-Supervised Models in Speech Enhancement
- 2021Efficient Personalized Speech Enhancement through Self-Supervised Learning
- 2022Joint Training of Speech Enhancement and Self-supervised Model for Noise-robust ASR
- 2026Masked Autoencoders as Universal Speech Enhancer
- 2022Investigating self-supervised learning for speech enhancement and separation
- 2021Self-Supervised Learning based Monaural Speech Enhancement with Complex-Cycle-Consistent