# Self Multi-Head Attention for Speaker Recognition

arXiv: https://arxiv.org/abs/1906.09890 · HTML: https://mlsanity.com/paper/1906.09890/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 13 of 13)

- [A General Survey on Attention Mechanisms in Deep Learning](https://mlsanity.com/paper/2203.14263/) (2022)
- [Local Multi-Head Channel Self-Attention for Facial Expression Recognition](https://mlsanity.com/paper/2111.07224/) (2021)
- [Serialized Multi-Layer Multi-Head Attention for Neural Speaker Embedding](https://mlsanity.com/paper/2107.06493/) (2021)
- [MACCIF-TDNN: Multi aspect aggregation of channel and context interdependence features in TDNN-based speaker verification](https://mlsanity.com/paper/2107.03104/) (2021)
- [Pretraining Approaches for Spoken Language Recognition: TalTech Submission to the OLR 2021 Challenge](https://mlsanity.com/paper/2205.07083/) (2022)
- [DRASP: A Dual-Resolution Attentive Statistics Pooling Framework for Automatic MOS Prediction](https://mlsanity.com/paper/2508.21407/) (2025)
- [Speaker embeddings by modeling channel-wise correlations](https://mlsanity.com/paper/2104.02571/) (2021)
- [Class Token and Knowledge Distillation for Multi-head Self-Attention Speaker Verification Systems](https://mlsanity.com/paper/2111.03842/) (2021)
- [An Effective Transformer-based Contextual Model and Temporal Gate Pooling for Speaker Identification](https://mlsanity.com/paper/2308.11241/) (2023)
- [Teffic-Audio: Tell Fact from Fiction](https://mlsanity.com/paper/2607.28351/) (2026)
- [Memory-Efficient Training for Text-Dependent SV with Independent Pre-trained Models](https://mlsanity.com/paper/2411.10828/) (2024)
- [Memory-Efficient Training for Deep Speaker Embedding Learning in Speaker Verification](https://mlsanity.com/paper/2412.01195/) (2024)
- [Anuvani-LID: Robust and Calibrated Spoken Language Identification for Indian Languages](https://mlsanity.com/paper/2610.20263/) (2026)

## Abstract (arXiv)

Most state-of-the-art Deep Learning (DL) approaches for speaker recognition work on a short utterance level. Given the speech signal, these algorithms extract a sequence of speaker embeddings from short segments and those are averaged to obtain an utterance level speaker representation. In this work we propose the use of an attention mechanism to obtain a discriminative speaker embedding given non fixed length speech utterances. Our system is based on a Convolutional Neural Network (CNN) that encodes short-term speaker features from the spectrogram and a self multi-head attention model that maps these representations into a long-term speaker embedding. The attention model that we propose produces multiple alignments from different subsegments of the CNN encoded states over the sequence. Hence this mechanism works as a pooling layer which decides the most discriminative features over the sequence to obtain an utterance level representation. We have tested this approach for the verification task for the VoxCeleb1 dataset. The results show that self multi-head attention outperforms both temporal and statistical pooling methods with a 18% of relative EER. Obtained results show a 58% relative improvement in EER compared to i-vector+PLDA.