mlsanityData to 19 Oct 2026

Self-Supervised Learning based Monaural Speech Enhancement with Complex-Cycle-Consistent

arXiv 2112.11142 ↗First posted 21 Dec 2021cs.SD eess.AScited by 0

On the map: Speech and audio language models → Speech enhancement and noise suppression

Abstract (arXiv)

Recently, self-supervised learning (SSL) techniques have been introduced to solve the monaural speech enhancement problem. Due to the lack of using clean phase information, the enhancement performance is limited in most SSL methods. Therefore, in this paper, we propose a phase-aware self-supervised learning based monaural speech enhancement method. The latent representations of both amplitude and phase are studied in two decoders of the foundation autoencoder (FAE) with only a limited set of clean speech signals independently. Then, the downstream autoencoder (DAE) learns a shared latent space between the clean speech and mixture representations with a large number of unseen mixtures. A complex-cycle-consistent (CCC) mechanism is proposed to minimize the reconstruction loss between the amplitude and phase domains. Besides, it is noticed that if the speech features are extracted as the multi-resolution spectra, the desired information distributed in spectra of different scales can be studied to further boost the performance. The NOISEX and DAPS corpora are used to generate mixtures with different interferences to evaluate the efficacy of the proposed method. It is highlighted that the clean speech and mixtures fed in FAE and DAE are not paired. Both ablation and comparison experimental results show that the proposed method clearly outperforms the state-of-the-art approaches.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. Builds on 24 of its 42 references that are in the map. First posted on arXiv 21 Dec 2021.

Builds on

24 of 42 references.

Closest ancestors · 5

Same direction, newest first

  1. 2021Domain Adaptation and Autoencoder Based Unsupervised Speech Enhancement9 Dec
  2. 2021A Two-stage Complex Network using Cycle-consistent Generative Adversarial Networks for Speech Enhancement5 Sep
  3. 2021On The Compensation Between Magnitude and Phase in Speech Separation11 Aug
  4. 2021Personalized Speech Enhancement through Self-Supervised Data Augmentation and Purification5 Apr
  5. 2021Speech enhancement with weakly labelled data from AudioSet19 Feb

Other references · 19

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  2. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 425
  3. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 389
  4. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 219
  5. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 211
  6. 2020Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separationcited by 105
  7. 2020Guided Variational Autoencoder for Disentanglement Learningcited by 59
  8. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 52
  9. 2021Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separationcited by 49
  10. 2020PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Losscited by 42
  11. 2020VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Networkcited by 32
  12. 2018CR-GAN: Learning Complete Representations for Multi-view Generationcited by 24
7 more
  1. 2020Self-supervised Learning for Speech Enhancementcited by 18
  2. 2020Ultra-Lightweight Speech Separation via Group Communicationcited by 13
  3. 2018Cycle-Consistent Speech Enhancementcited by 11
  4. 2020Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-trainingcited by 10
  5. 2020Self-Supervised Learning from Contrastive Mixtures for Personalized Speech Enhancementcited by 7
  6. 2021Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuningcited by 6
  7. 2021A Multi-Resolution Front-End for End-to-End Speech Anti-Spoofingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Self-Supervised Learning based Monaural Speech Enhancement with Multi-Task Pre-Training
  2. 2024Complex-Cycle-Consistent Diffusion Model for Monaural Speech Enhancement
  3. 2022Feature Learning and Ensemble Pre-Tasks Based Self-Supervised Speech Denoising and Dereverberation
  4. 2021Efficient Personalized Speech Enhancement through Self-Supervised Learning
  5. 2022Investigating self-supervised learning for speech enhancement and separation
  6. 2023D2Former: A Fully Complex Dual-Path Dual-Decoder Conformer Network using Joint Complex Masking and Complex Spectral Mapping for Monaural Speech Enhancement
  7. 2026Masked Autoencoders as Universal Speech Enhancer
  8. 2022DBT-Net: Dual-branch federative magnitude and phase estimation with attention-in-attention transformer for monaural speech enhancement