# Learning from human perception to improve automatic speaker verification in style-mismatched conditions

arXiv: https://arxiv.org/abs/2206.13684 · HTML: https://mlsanity.com/paper/2206.13684/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [Attention-Based Models for Text-Dependent Speaker Verification](https://mlsanity.com/paper/1710.10470/) (2017)
- [ResNeXt and Res2Net Structures for Speaker Verification](https://mlsanity.com/paper/2007.02480/) (2020)
- [Angular Softmax Loss for End-to-end Speaker Verification](https://mlsanity.com/paper/1806.03464/) (2018)
- [Exploring the Use of an Unsupervised Autoregressive Model as a Shared Encoder for Text-Dependent Speaker Verification](https://mlsanity.com/paper/2008.03615/) (2020)
- [Variable frame rate-based data augmentation to handle speaking-style variability for automatic speaker verification](https://mlsanity.com/paper/2008.03616/) (2020)
- [Optimization of the Area Under the ROC Curve using Neural Network Supervectors for Text-Dependent Speaker Verification](https://mlsanity.com/paper/1901.11332/) (2019)
- [Speaker discrimination in humans and machines: Effects of speaking style variability](https://mlsanity.com/paper/2008.03617/) (2020)

## Built on by (most cited first, 2 of 2)

- [The Lipreading Gap: Do VSR Models Perceive Visual Speech Like Human Lipreaders?](https://mlsanity.com/paper/2606.07435/) (2026)
- [Effects of language mismatch in automatic forensic voice comparison using deep learning embeddings](https://mlsanity.com/paper/2209.12602/) (2022)

## Abstract (arXiv)

Our prior experiments show that humans and machines seem to employ different approaches to speaker discrimination, especially in the presence of speaking style variability. The experiments examined read versus conversational speech. Listeners focused on speaker-specific idiosyncrasies while "telling speakers together", and on relative distances in a shared acoustic space when "telling speakers apart". However, automatic speaker verification (ASV) systems use the same loss function irrespective of target or non-target trials. To improve ASV performance in the presence of style variability, insights learnt from human perception are used to design a new training loss function that we refer to as "CllrCE loss". CllrCE loss uses both speaker-specific idiosyncrasies and relative acoustic distances between speakers to train the ASV system. When using the UCLA speaker variability database, in the x-vector and conditioning setups, CllrCE loss results in significant relative improvements in EER by 1-66%, and minDCF by 1-31% and 1-56%, respectively, when compared to the x-vector baseline. Using the SITW evaluation tasks, which involve different conversational speech tasks, the proposed loss combined with self-attention conditioning results in significant relative improvements in EER by 2-5% and minDCF by 6-12% over baseline. In the SITW case, performance improvements were consistent only with conditioning.