# Attention-based conditioning methods using variable frame rate for style-robust speaker verification

arXiv: https://arxiv.org/abs/2206.13680 · HTML: https://mlsanity.com/paper/2206.13680/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Attentive Statistics Pooling for Deep Speaker Embedding](https://mlsanity.com/paper/1803.10963/) (2018)
- [ResNeXt and Res2Net Structures for Speaker Verification](https://mlsanity.com/paper/2007.02480/) (2020)
- [Attention-based Conditioning Methods for External Knowledge Integration](https://mlsanity.com/paper/1906.03674/) (2019)
- [End-to-End Attention based Text-Dependent Speaker Verification](https://mlsanity.com/paper/1701.00562/) (2017)
- [Exploring the Use of an Unsupervised Autoregressive Model as a Shared Encoder for Text-Dependent Speaker Verification](https://mlsanity.com/paper/2008.03615/) (2020)
- [Variable frame rate-based data augmentation to handle speaking-style variability for automatic speaker verification](https://mlsanity.com/paper/2008.03616/) (2020)
- [Serialized Multi-Layer Multi-Head Attention for Neural Speaker Embedding](https://mlsanity.com/paper/2107.06493/) (2021)
- [Attention Mechanism in Speaker Recognition: What Does It Learn in Deep Speaker Embedding?](https://mlsanity.com/paper/1809.09311/) (2018)
- [Speaker discrimination in humans and machines: Effects of speaking style variability](https://mlsanity.com/paper/2008.03617/) (2020)

## Abstract (arXiv)

We propose an approach to extract speaker embeddings that are robust to speaking style variations in text-independent speaker verification. Typically, speaker embedding extraction includes training a DNN for speaker classification and using the bottleneck features as speaker representations. Such a network has a pooling layer to transform frame-level to utterance-level features by calculating statistics over all utterance frames, with equal weighting. However, self-attentive embeddings perform weighted pooling such that the weights correspond to the importance of the frames in a speaker classification task. Entropy can capture acoustic variability due to speaking style variations. Hence, an entropy-based variable frame rate vector is proposed as an external conditioning vector for the self-attention layer to provide the network with information that can address style effects. This work explores five different approaches to conditioning. The best conditioning approach, concatenation with gating, provided statistically significant improvements over the x-vector baseline in 12/23 tasks and was the same as the baseline in 11/23 tasks when using the UCLA speaker variability database. It also significantly outperformed self-attention without conditioning in 9/23 tasks and was worse in 1/23. The method also showed significant improvements in multi-speaker scenarios of SITW.