mlsanityData to 14 Oct 2026

Feature Normalization for Fine-tuning Self-Supervised Models in Speech Enhancement

arXiv 2306.08406 ↗First posted 14 Jun 2023eess.AS cs.LG cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Large, pre-trained representation models trained using self-supervised learning have gained popularity in various fields of machine learning because they are able to extract high-quality salient features from input data. As such, they have been frequently used as base networks for various pattern classification tasks such as speech recognition. However, not much research has been conducted on applying these types of models to the field of speech signal generation. In this paper, we investigate the feasibility of using pre-trained speech representation models for a downstream speech enhancement task. To alleviate mismatches between the input features of the pre-trained model and the target enhancement model, we adopt a novel feature normalization technique to smoothly link these modules together. Our proposed method enables significant improvements in speech quality compared to baselines when combined with various types of pre-trained speech models.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 34 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

25 of 34 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  7. 2020Self-supervised Learning: Generative or Contrastivecited by 694
  8. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  9. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 340
  10. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  11. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 188
  12. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 186
13 more
  1. 2020Real Time Speech Enhancement in the Waveform Domaincited by 186
  2. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 151
  3. 2020Multi-task self-supervised learning for Robust Speech Recognitioncited by 130
  4. 2018Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separationcited by 125
  5. 2018Domain Adversarial Training for Accented Speech Recognitioncited by 40
  6. 2022Investigating self-supervised learning for speech enhancement and separationcited by 35
  7. 2020Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learningcited by 33
  8. 2021Large-scale ASR Domain Adaptation using Self- and Semi-supervised Learningcited by 26
  9. 2022Boosting Self-Supervised Embeddings for Speech Enhancementcited by 24
  10. 2022DRAFT: A Novel Framework to Reduce Domain Shifting in Self-supervised Learning and Its Application to Children's ASRcited by 21
  11. 2021Don't speak too fast: The impact of data bias on self-supervised speech modelscited by 20
  12. 2018Noise Adaptive Speech Enhancement using Domain Adversarial Trainingcited by 15
  13. 2021Self-Supervised Learning Based Domain Adaptation for Robust Speaker Verificationcited by 9

Built on by

Not cited yet.