mlsanityData to 7 Oct 2026

Investigating self-supervised learning for speech enhancement and separation

arXiv 2203.07960 ↗First posted 15 Mar 2022eess.AScited by 35

Not on the map. Why?

Abstract (arXiv)

Speech enhancement and separation are two fundamental tasks for robust speech processing. Speech enhancement suppresses background noise while speech separation extracts target speech from interfering speakers. Despite a great number of supervised learning-based enhancement and separation methods having been proposed and achieving good performance, studies on applying self-supervised learning (SSL) to enhancement and separation are limited. In this paper, we evaluate 13 SSL upstream methods on speech enhancement and separation downstream tasks. Our experimental results on Voicebank-DEMAND and Libri2Mix show that some SSL representations consistently outperform baseline features including the short-time Fourier transform (STFT) magnitude and log Mel filterbank (FBANK). Furthermore, we analyze the factors that make existing SSL frameworks difficult to apply to speech enhancement and separation and discuss the representation properties desired for both tasks. Our study is included as the official speech enhancement and separation downstreams for SUPERB.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Mar 2022.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 8; 2024: 8; 2025: 7; 2026: 6.

35 papers.

Sort:

By area:

  1. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 38
  2. 2023DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Modelscited by 32
  3. 2023Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understandingcited by 29
  4. 2025AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancementcited by 23
  5. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 17
  6. 2023Audio-Visual Speech Enhancement with Score-Based Generative Modelscited by 12
  7. 2023Adapting Multi-Lingual ASR Models for Handling Multiple Talkerscited by 11
  8. 2024TRAMBA: A Hybrid Transformer and Mamba Architecture for Practical Audio and Bone Conduction Speech Super Resolution and Enhancement on Mobile and Wearable Platformscited by 11
  9. 2022Self-Supervised Learning for Speech Enhancement through Synthesiscited by 10
  10. 2024BSS-CFFMA: Cross-Domain Feature Fusion and Multi-Attention Speech Enhancement Network based on Self-Supervised Embeddingcited by 9
  11. 2025From Faces to Voices: Learning Hierarchical Representations for High-quality Video-to-Speechcited by 9
  12. 2022MTI-Net: A Multi-Target Speech Intelligibility Prediction Modelcited by 6
23 more
  1. 2024Music Foundation Model as Generic Booster for Music Downstream Taskscited by 6
  2. 2022Speech separation with large-scale self-supervised learningcited by 5
  3. 2023AV2Wav: Diffusion-Based Re-synthesis from Continuous Self-supervised Features for Audio-Visual Speech Enhancementcited by 5
  4. 2025PASE: Leveraging the Phonological Prior of WavLM for Low-Hallucination Generative Speech Enhancementcited by 5
  5. 2022Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separationcited by 3
  6. 2022Deep neural network techniques for monaural speech enhancement: state of the art analysiscited by 3
  7. 2024Explaining Relationships Among Research Paperscited by 3
  8. 2025StableQuant: Layer Adaptive Post-Training Quantization for Speech Foundation Modelscited by 3
  9. 2022Handling Trade-Offs in Speech Separation with Sparsely-Gated Mixture of Expertscited by 2
  10. 2025DiceHuBERT: Distilling HuBERT with a Self-Supervised Learning Objectivecited by 2
  11. 2024AnnoTheia: A Semi-Automatic Annotation Toolkit for Audio-Visual Speech Technologiescited by 1
  12. 2024A Closer Look at Wav2Vec2 Embeddings for On-Device Single-Channel Speech Enhancementcited by 1
  13. 2024Heterogeneous Space Fusion and Dual-Dimension Attention: A New Paradigm for Speech Enhancementcited by 1
  14. 2023Speech Separation based on Contrastive Learning and Deep Modularizationcited by 0
  15. 2023Feature Normalization for Fine-tuning Self-Supervised Models in Speech Enhancementcited by 0
  16. 2025Magnitude-Phase Dual-Path Speech Enhancement Network based on Self-Supervised Embedding and Perceptual Contrast Stretch Boostingcited by 0
  17. 2025Causal Self-supervised Pretrained Frontend with Predictive Code for Speech Separationcited by 0
  18. 2026Masked Autoencoders as Universal Speech Enhancercited by 0
  19. 2026Towards Lightweight Adaptation of Speech Enhancement Models in Real-World Environmentscited by 0
  20. 2026MSR-HuBERT: Self-supervised Pre-training for Adaptation to Multiple Sampling Ratescited by 0
  21. 2026UniPASE: A Generative Model for Universal Speech Enhancement with High Fidelity and Low Hallucinationscited by 0
  22. 2026PhASE-Flow: Phonetic-Conditioned Acoustic Flow Matching in SSL Representation Domain for Speech Enhancementcited by 0
  23. 2026Bridging Self-Supervised Learning and Speech Enhancement: A Wav2Vec2-Conditioned Frameworkcited by 0