# VoiceID Loss: Speech Enhancement for Speaker Verification

arXiv: https://arxiv.org/abs/1904.03601 · HTML: https://mlsanity.com/paper/1904.03601/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2019.

## Built on by (most cited first, 11 of 11)

- [Thunder : Unified Regression-Diffusion Speech Enhancement with a Single Reverse Step using Brownian Bridge](https://mlsanity.com/paper/2406.06139/) (2024)
- [Dual-Branch Knowledge Distillation for Noise-Robust Synthetic Speech Detection](https://mlsanity.com/paper/2310.08869/) (2023)
- [xLSTM-SENet: xLSTM for Single-Channel Speech Enhancement](https://mlsanity.com/paper/2501.06146/) (2025)
- [PL-EESR: Perceptual Loss Based END-TO-END Robust Speaker Representation Extraction](https://mlsanity.com/paper/2110.00940/) (2021)
- [Diff-SV: A Unified Hierarchical Framework for Noise-Robust Speaker Verification Using Score-Based Diffusion Probabilistic Models](https://mlsanity.com/paper/2309.08320/) (2023)
- [DRASP: A Dual-Resolution Attentive Statistics Pooling Framework for Automatic MOS Prediction](https://mlsanity.com/paper/2508.21407/) (2025)
- [Learnable Nonlinear Compression for Robust Speaker Verification](https://mlsanity.com/paper/2202.05236/) (2022)
- [Noise-Conditioned Mixture-of-Experts Framework for Robust Speaker Verification](https://mlsanity.com/paper/2510.18533/) (2025)
- [Task-aware Warping Factors in Mask-based Speech Enhancement](https://mlsanity.com/paper/2108.12128/) (2021)
- [SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points](https://mlsanity.com/paper/2111.04436/) (2021)
- [LC4SV: A Denoising Framework Learning to Compensate for Unseen Speaker Verification Models](https://mlsanity.com/paper/2311.16604/) (2023)

## Abstract (arXiv)

In this paper, we propose VoiceID loss, a novel loss function for training a speech enhancement model to improve the robustness of speaker verification. In contrast to the commonly used loss functions for speech enhancement such as the L2 loss, the VoiceID loss is based on the feedback from a speaker verification model to generate a ratio mask. The generated ratio mask is multiplied pointwise with the original spectrogram to filter out unnecessary components for speaker verification. In the experiments, we observed that the enhancement network, after training with the VoiceID loss, is able to ignore a substantial amount of time-frequency bins, such as those dominated by noise, for verification. The resulting model consistently improves the speaker verification system on both clean and noisy conditions.