mlsanityData to 19 Oct 2026

Generic Speech Enhancement with Self-Supervised Representation Space Loss

arXiv 2507.07631 ↗First posted 10 Jul 2025eess.AS cs.SD eess.SPcited by 2

On the map: Speech and audio language models → Speech enhancement and noise suppression

Abstract (arXiv)

Single-channel speech enhancement is utilized in various tasks to mitigate the effect of interfering signals. Conventionally, to ensure the speech enhancement performs optimally, the speech enhancement has needed to be tuned for each task. Thus, generalizing speech enhancement models to unknown downstream tasks has been challenging. This study aims to construct a generic speech enhancement front-end that can improve the performance of back-ends to solve multiple downstream tasks. To this end, we propose a novel training criterion that minimizes the distance between the enhanced and the ground truth clean signal in the feature representation domain of self-supervised learning models. Since self-supervised learning feature representations effectively express high-level speech information useful for solving various downstream tasks, the proposal is expected to make speech enhancement models preserve such information. Experimental validation demonstrates that the proposal improves the performance of multiple speech tasks while maintaining the perceptual quality of the enhanced signal.

Key facts. Cited by 2 papers in the map (data through 19 Oct 2026). Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. Builds on 56 of its 84 references that are in the map. First posted on arXiv 10 Jul 2025.

Builds on

56 of 84 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Rethinking Processing Distortions: Disentangling the Impact of Speech Enhancement Errors on Speech Recognition Performance23 Apr
  2. 2023Downstream Task Agnostic Speech Enhancement with Self-Supervised Representation Loss24 May
  3. 2022ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding19 Jul
  4. 2022FRCRN: Boosting Feature Representation using Frequency Recurrence for Monaural Speech Enhancement15 Jun
  5. 2022Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation2 Apr
  6. 2022End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation1 Apr
  7. 2022How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR18 Jan
  8. 2022Learning to Enhance or Not: Neural Network-Based Switching of Enhanced and Observed Signals for Overlapping Speech Recognition11 Jan
  9. 2021A Conformer-based ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement and Speech Separation18 Nov
  10. 2021Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition2 Jun
  11. 2021TSTNN: Two-stage Transformer based Neural Network for Speech Enhancement in the Time Domain18 Mar

Other references · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,923
  2. 2014Adam: A Method for Stochastic Optimizationcited by 38,210
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,762
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 2,135
  7. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 793
  8. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 754
  9. 2020Self-supervised Learning: Generative or Contrastivecited by 701
  10. 2018SDR - half-baked or well done?cited by 626
  11. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 596
  12. 2020A Survey on Contrastive Self-supervised Learningcited by 558
33 more
  1. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 390
  2. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 345
  3. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 301
  4. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 292
  5. 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 284
  6. 2020Real Time Speech Enhancement in the Waveform Domaincited by 282
  7. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 230
  8. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 211
  9. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 211
  10. 2023End-to-End Speech Recognition: A Surveycited by 163
  11. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 159
  12. 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 122
  13. 2020FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancementcited by 121
  14. 2022ICASSP 2022 Deep Noise Suppression Challengecited by 107
  15. 2023ICASSP 2023 Deep Noise Suppression Challengecited by 101
  16. 2021Interspeech 2021 Deep Noise Suppression Challengecited by 96
  17. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 92
  18. 2022TF-GridNet: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separationcited by 76
  19. 2021The Zero Resource Speech Challenge 2021: Spoken language modellingcited by 55
  20. 2020VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognitioncited by 54
  21. 2020Improving speaker discrimination of target speech extraction with time-domain SpeakerBeamcited by 45
  22. 2020Improving noise robust automatic speech recognition with single-channel time-domain enhancement networkcited by 45
  23. 2020Weighted Speech Distortion Losses for Neural-network-based Real-time Speech Enhancementcited by 39
  24. 2018Building state-of-the-art distant speech recognition using the CHiME-4 challenge with a setup of speech enhancement baselinecited by 30
  25. 2023Multi-resolution HuBERT: Multi-resolution Speech Self-Supervised Learning with Masked Unit Predictioncited by 28
  26. 2019VoiceID Loss: Speech Enhancement for Speaker Verificationcited by 27
  27. 2023VoxSRC 2022: The Fourth VoxCeleb Speaker Recognition Challengecited by 22
  28. 2020Dual Application of Speech Enhancement for Automatic Speech Recognitioncited by 18
  29. 2022End-to-End Integration of Speech Recognition, Dereverberation, Beamforming, and Self-Supervised Learning Representationcited by 18
  30. 2023Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learningcited by 11
  31. 2024Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapterscited by 8
  32. 2024SpeakerBeam-SS: Real-time Target Speaker Extraction with Lightweight Conv-TasNet and State Space Modelingcited by 8
  33. 2024MS-HuBERT: Mitigating Pre-training and Inference Mismatch in Masked Language Modelling methods for learning Speech Representationscited by 3

Built on by

Cited by year (2026 to 19 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026DBMIF: a deep balanced multimodal iterative fusion framework for air- and bone-conduction speech enhancementcited by 0
  2. 2026Transcript-Supervised Post-Training of Generative Speech Enhancement on Real Recordings via Reinforce Adjoint Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Feature Normalization for Fine-tuning Self-Supervised Models in Speech Enhancement
  2. 2023The Effect of Spoken Language on Speech Enhancement using Self-Supervised Speech Representation Loss Functions
  3. 2023Perceive and predict: self-supervised speech representation based loss functions for speech enhancement
  4. 2022Investigating self-supervised learning for speech enhancement and separation
  5. 2022Feature Learning and Ensemble Pre-Tasks Based Self-Supervised Speech Denoising and Dereverberation
  6. 2021Self-Supervised Learning based Monaural Speech Enhancement with Multi-Task Pre-Training
  7. 2022Boosting Self-Supervised Embeddings for Speech Enhancement
  8. 2021Efficient Personalized Speech Enhancement through Self-Supervised Learning