# A Systematic Comparison of Phonetic Aware Techniques for Speech Enhancement

arXiv: https://arxiv.org/abs/2206.11000 · HTML: https://mlsanity.com/paper/2206.11000/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 22 Jun 2022.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [Generative Spoken Language Modeling from Raw Audio](https://mlsanity.com/paper/2102.01192/) (2021)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [Music Source Separation in the Waveform Domain](https://mlsanity.com/paper/1911.13254/) (2019)
- [A scalable noisy speech dataset and online subjective test framework](https://mlsanity.com/paper/1909.08050/) (2019)
- [A Wavenet for Speech Denoising](https://mlsanity.com/paper/1706.07162/) (2017)
- [Improving GANs for Speech Enhancement](https://mlsanity.com/paper/2001.05532/) (2020)
- [Multi-talker ASR for an unknown number of sources: Joint training of source counting, separation and ASR](https://mlsanity.com/paper/2006.02786/) (2020)
- [Probability density distillation with generative adversarial networks for high-quality parallel waveform generation](https://mlsanity.com/paper/1904.04472/) (2019)
- [Guided Source Separation Meets a Strong ASR Backend: Hitachi/Paderborn University Joint Investigation for Dinner Party ASR](https://mlsanity.com/paper/1905.12230/) (2019)
- [Single channel voice separation for unknown number of speakers under reverberant and noisy settings](https://mlsanity.com/paper/2011.02329/) (2020)
- [An Investigation into the Effectiveness of Enhancement in ASR Training and Test for CHiME-5 Dinner Party Transcription](https://mlsanity.com/paper/1909.12208/) (2019)
- [Continual self-training with bootstrapped remixing for speech enhancement](https://mlsanity.com/paper/2110.10103/) (2021)
- [An individualized super Gaussian single microphone Speech Enhancement for hearing aid users with smartphone as an assistive device](https://mlsanity.com/paper/1812.03916/) (2018)
- [Unpaired Speech Enhancement by Acoustic and Adversarial Supervision for Speech Recognition](https://mlsanity.com/paper/1811.02182/) (2018)

## Built on by (most cited first, 5 of 5)

- [Perceive and predict: self-supervised speech representation based loss functions for speech enhancement](https://mlsanity.com/paper/2301.04388/) (2023)
- [PAST: Phonetic-Acoustic Speech Tokenizer](https://mlsanity.com/paper/2505.14470/) (2025)
- [The Effect of Spoken Language on Speech Enhancement using Self-Supervised Speech Representation Loss Functions](https://mlsanity.com/paper/2307.14502/) (2023)
- [PAAPLoss: A Phonetic-Aligned Acoustic Parameter Loss for Speech Enhancement](https://mlsanity.com/paper/2302.08095/) (2023)
- [A Closer Look at Wav2Vec2 Embeddings for On-Device Single-Channel Speech Enhancement](https://mlsanity.com/paper/2403.01369/) (2024)

## Abstract (arXiv)

Speech enhancement has seen great improvement in recent years using end-to-end neural networks. However, most models are agnostic to the spoken phonetic content. Recently, several studies suggested phonetic-aware speech enhancement, mostly using perceptual supervision. Yet, injecting phonetic features during model optimization can take additional forms (e.g., model conditioning). In this paper, we conduct a systematic comparison between different methods of incorporating phonetic information in a speech enhancement model. By conducting a series of controlled experiments, we observe the influence of different phonetic content models as well as various feature-injection techniques on enhancement performance, considering both causal and non-causal models. Specifically, we evaluate three settings for injecting phonetic information, namely: i) feature conditioning; ii) perceptual supervision; and iii) regularization. Phonetic features are obtained using an intermediate layer of either a supervised pre-trained Automatic Speech Recognition (ASR) model or by using a pre-trained Self-Supervised Learning (SSL) model. We further observe the effect of choosing different embedding layers on performance, considering both manual and learned configurations. Results suggest that using a SSL model as phonetic features outperforms the ASR one in most cases. Interestingly, the conditioning setting performs best among the evaluated configurations.