# DT-SV: A Transformer-based Time-domain Approach for Speaker Verification

arXiv: https://arxiv.org/abs/2205.13249 · HTML: https://mlsanity.com/paper/2205.13249/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [In defence of metric learning for speaker recognition](https://mlsanity.com/paper/2003.11982/) (2020)
- [Attentive Statistics Pooling for Deep Speaker Embedding](https://mlsanity.com/paper/1803.10963/) (2018)
- [CN-CELEB: a challenging Chinese speaker recognition dataset](https://mlsanity.com/paper/1911.01799/) (2019)
- [Adversarial defense for automatic speaker verification by cascaded self-supervised learning models](https://mlsanity.com/paper/2102.07047/) (2021)
- [Improved RawNet with Feature Map Scaling for Text-independent Speaker Verification using Raw Waveforms](https://mlsanity.com/paper/2004.00526/) (2020)
- [Federated Learning with Dynamic Transformer for Text to Speech](https://mlsanity.com/paper/2107.08795/) (2021)
- [Self-Supervised Learning Based Domain Adaptation for Robust Speaker Verification](https://mlsanity.com/paper/2108.13843/) (2021)
- [S-vectors and TESA: Speaker Embeddings and a Speaker Authenticator Based on Transformer Encoder](https://mlsanity.com/paper/2008.04659/) (2020)
- [Evolutionary Algorithm Enhanced Neural Architecture Search for Text-Independent Speaker Verification](https://mlsanity.com/paper/2008.05695/) (2020)
- [Serialized Multi-Layer Multi-Head Attention for Neural Speaker Embedding](https://mlsanity.com/paper/2107.06493/) (2021)
- [Double Multi-Head Attention for Speaker Verification](https://mlsanity.com/paper/2007.13199/) (2020)
- [How to Improve Your Speaker Embeddings Extractor in Generic Toolkits](https://mlsanity.com/paper/1811.02066/) (2018)
- [Small footprint Text-Independent Speaker Verification for Embedded Systems](https://mlsanity.com/paper/2011.01709/) (2020)

## Built on by (most cited first, 3 of 3)

- [Exploring Transformers for Behavioural Biometrics: A Case Study in Gait Recognition](https://mlsanity.com/paper/2206.01441/) (2022)
- [Improving Transformer-based Networks With Locality For Automatic Speaker Verification](https://mlsanity.com/paper/2302.08639/) (2023)
- [Adaptive Sparse and Monotonic Attention for Transformer-based Automatic Speech Recognition](https://mlsanity.com/paper/2209.15176/) (2022)

## Abstract (arXiv)

Speaker verification (SV) aims to determine whether the speaker's identity of a test utterance is the same as the reference speech. In the past few years, extracting speaker embeddings using deep neural networks for SV systems has gone mainstream. Recently, different attention mechanisms and Transformer networks have been explored widely in SV fields. However, utilizing the original Transformer in SV directly may have frame-level information waste on output features, which could lead to restrictions on capacity and discrimination of speaker embeddings. Therefore, we propose an approach to derive utterance-level speaker embeddings via a Transformer architecture that uses a novel loss function named diffluence loss to integrate the feature information of different Transformer layers. Therein, the diffluence loss aims to aggregate frame-level features into an utterance-level representation, and it could be integrated into the Transformer expediently. Besides, we also introduce a learnable mel-fbank energy feature extractor named time-domain feature extractor that computes the mel-fbank features more precisely and efficiently than the standard mel-fbank extractor. Combining Diffluence loss and Time-domain feature extractor, we propose a novel Transformer-based time-domain SV model (DT-SV) with faster training speed and higher accuracy. Experiments indicate that our proposed model can achieve better performance in comparison with other models.