# A Framework for Robust Speaker Verification in Highly Noisy Environments Leveraging Both Noisy and Enhanced Audio

arXiv: https://arxiv.org/abs/2508.18913 · HTML: https://mlsanity.com/paper/2508.18913/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 26 Aug 2025.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 11 of 11)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Speaker Recognition Based on Deep Learning: An Overview](https://mlsanity.com/paper/2012.00931/) (2020)
- [DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement](https://mlsanity.com/paper/2305.08227/) (2023)
- [VoiceID Loss: Speech Enhancement for Speaker Verification](https://mlsanity.com/paper/1904.03601/) (2019)
- [SpeakerNet: 1D Depth-wise Separable Convolutional Network for Text-Independent Speaker Recognition and Verification](https://mlsanity.com/paper/2010.12653/) (2020)
- [Within-sample variability-invariant loss for robust speaker recognition under noisy environments](https://mlsanity.com/paper/2002.00924/) (2020)
- [Feature Enhancement with Deep Feature Losses for Speaker Verification](https://mlsanity.com/paper/1910.11905/) (2019)
- [Extended U-Net for Speaker Verification in Noisy Environments](https://mlsanity.com/paper/2206.13044/) (2022)
- [LC4SV: A Denoising Framework Learning to Compensate for Unseen Speaker Verification Models](https://mlsanity.com/paper/2311.16604/) (2023)

## Built on by (most cited first, 1 of 1)

- [LIFT-SE: Linguistic Inference Followed by Flow Transformation for Generative Speech Enhancement](https://mlsanity.com/paper/2610.09963/) (2026)

## Abstract (arXiv)

Recent advancements in speaker verification techniques show promise, but their performance often deteriorates significantly in challenging acoustic environments. Although speech enhancement methods can improve perceived audio quality, they may unintentionally distort speaker-specific information, which can affect verification accuracy. This problem has become more noticeable with the increasing use of generative deep neural networks (DNNs) for speech enhancement. While these networks can produce intelligible speech even in conditions of very low signal-to-noise ratio (SNR), they may also severely alter distinctive speaker characteristics. To tackle this issue, we propose a novel neural network framework that effectively combines speaker embeddings extracted from both noisy and enhanced speech using a Siamese architecture. This architecture allows us to leverage complementary information from both sources, enhancing the robustness of speaker verification under severe noise conditions. Our framework is lightweight and agnostic to specific speaker verification and speech enhancement techniques, enabling the use of a wide range of state-of-the-art solutions without modification. Experimental results demonstrate the superior performance of our proposed framework.