# STC speaker recognition systems for the NIST SRE 2021

arXiv: https://arxiv.org/abs/2111.02298 · HTML: https://mlsanity.com/paper/2111.02298/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 3 Nov 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [RetinaFace: Single-stage Dense Face Localisation in the Wild](https://mlsanity.com/paper/1905.00641/) (2019)
- [Attentive Statistics Pooling for Deep Speaker Embedding](https://mlsanity.com/paper/1803.10963/) (2018)
- [Toward Transformer-Based Object Detection](https://mlsanity.com/paper/2012.09958/) (2020)
- [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.07698/) (2018)
- [BUT System Description to VoxCeleb Speaker Recognition Challenge 2019](https://mlsanity.com/paper/1910.12592/) (2019)
- [Attentive Normalization](https://mlsanity.com/paper/1908.01259/) (2019)
- [The BOSARIS Toolkit: Theory, Algorithms and Code for Surviving the New DCF](https://mlsanity.com/paper/1304.2865/) (2013)
- [Deep Speaker Embeddings for Far-Field Speaker Recognition on Short Utterances](https://mlsanity.com/paper/2002.06033/) (2020)
- [Harnessing Unrecognizable Faces for Improving Face Recognition](https://mlsanity.com/paper/2106.04112/) (2021)
- [On deep speaker embeddings for text-independent speaker recognition](https://mlsanity.com/paper/1804.10080/) (2018)
- [Prototype Memory for Large-scale Face Representation Learning](https://mlsanity.com/paper/2105.02103/) (2021)

## Built on by (most cited first, 6 of 6)

- [Robust Speaker Recognition with Transformers Using wav2vec 2.0](https://mlsanity.com/paper/2203.15095/) (2022)
- [Universal speaker recognition encoders for different speech segments duration](https://mlsanity.com/paper/2210.16231/) (2022)
- [Investigation of Different Calibration Methods for Deep Speaker Embedding based Verification Systems](https://mlsanity.com/paper/2203.15106/) (2022)
- [Meta-Learning Framework for End-to-End Imposter Identification in Unseen Speaker Recognition](https://mlsanity.com/paper/2306.00952/) (2023)
- [Quality-Aware Prototype Memory for Face Representation Learning](https://mlsanity.com/paper/2311.07734/) (2023)
- [Post-Training Embedding Alignment for Decoupling Enrollment and Runtime Speaker Recognition Models](https://mlsanity.com/paper/2401.12440/) (2024)

## Abstract (arXiv)

This paper presents a description of STC Ltd. systems submitted to the NIST 2021 Speaker Recognition Evaluation for both fixed and open training conditions. These systems consists of a number of diverse subsystems based on using deep neural networks as feature extractors. During the NIST 2021 SRE challenge we focused on the training of the state-of-the-art deep speaker embeddings extractors like ResNets and ECAPA networks by using additive angular margin based loss functions. Additionally, inspired by the recent success of the wav2vec 2.0 features in automatic speech recognition we explored the effectiveness of this approach for the speaker verification filed. According to our observation the fine-tuning of the pretrained large wav2vec 2.0 model provides our best performing systems for open track condition. Our experiments with wav2vec 2.0 based extractors for the fixed condition showed that unsupervised autoregressive pretraining with Contrastive Predictive Coding loss opens the door to training powerful transformer-based extractors from raw speech signals. For video modality we developed our best solution with RetinaFace face detector and deep ResNet face embeddings extractor trained on large face image datasets. The final results for primary systems were obtained by different configurations of subsystems fusion on the score level followed by score calibration.