# Personalized Speech Enhancement Without a Separate Speaker Embedding Model

arXiv: https://arxiv.org/abs/2406.09928 · HTML: https://mlsanity.com/paper/2406.09928/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 14 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors](https://mlsanity.com/paper/2110.01763/) (2021)
- [ICASSP 2023 Deep Noise Suppression Challenge](https://mlsanity.com/paper/2303.11510/) (2023)
- [ICASSP 2022 Deep Noise Suppression Challenge](https://mlsanity.com/paper/2202.13288/) (2022)
- [VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition](https://mlsanity.com/paper/2009.04323/) (2020)
- [SpEx: Multi-Scale Time Domain Speaker Extraction Network](https://mlsanity.com/paper/2004.08326/) (2020)
- [Personalized Speech Enhancement: New Models and Comprehensive Evaluation](https://mlsanity.com/paper/2110.09625/) (2021)
- [A consolidated view of loss functions for supervised deep learning-based speech enhancement](https://mlsanity.com/paper/2009.12286/) (2020)
- [Personalized PercepNet: Real-time, Low-complexity Target Voice Separation and Enhancement](https://mlsanity.com/paper/2106.04129/) (2021)
- [TEA-PSE 3.0: Tencent-Ethereal-Audio-Lab Personalized Speech Enhancement System For ICASSP 2023 DNS Challenge](https://mlsanity.com/paper/2303.07704/) (2023)
- [AECMOS: A speech quality assessment metric for echo impairment](https://mlsanity.com/paper/2110.03010/) (2021)
- [DeepVQE: Real Time Deep Voice Quality Enhancement for Joint Acoustic Echo Cancellation, Noise Suppression and Dereverberation](https://mlsanity.com/paper/2306.03177/) (2023)
- [Zero-Shot Personalized Speech Enhancement through Speaker-Informed Model Selection](https://mlsanity.com/paper/2105.03542/) (2021)
- [Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separation](https://mlsanity.com/paper/2210.12635/) (2022)
- [The NPU-Elevoc Personalized Speech Enhancement System for ICASSP2023 DNS Challenge](https://mlsanity.com/paper/2303.06811/) (2023)
- [ICASSP 2023 Acoustic Echo Cancellation Challenge](https://mlsanity.com/paper/2309.12553/) (2023)
- [A Framework for Unified Real-time Personalized and Non-Personalized Speech Enhancement](https://mlsanity.com/paper/2302.11768/) (2023)

## Built on by (most cited first, 1 of 1)

- [Adapting Personalized Speech Enhancement for Low-Latency Audio-Visual Target-Speaker Extraction](https://mlsanity.com/paper/2609.30631/) (2026)

## Abstract (arXiv)

Personalized speech enhancement (PSE) models can improve the audio quality of teleconferencing systems by adapting to the characteristics of a speaker's voice. However, most existing methods require a separate speaker embedding model to extract a vector representation of the speaker from enrollment audio, which adds complexity to the training and deployment process. We propose to use the internal representation of the PSE model itself as the speaker embedding, thereby avoiding the need for a separate model. We show that our approach performs equally well or better than the standard method of using a pre-trained speaker embedding model on noise suppression and echo cancellation tasks. Moreover, our approach surpasses the ICASSP 2023 Deep Noise Suppression Challenge winner by 0.15 in Mean Opinion Score.