# The 2021 NIST Speaker Recognition Evaluation

arXiv: https://arxiv.org/abs/2204.10242 · HTML: https://mlsanity.com/paper/2204.10242/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 21 Apr 2022.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.07698/) (2018)
- [NIST SRE CTS Superset: A large-scale dataset for telephony speaker recognition](https://mlsanity.com/paper/2108.07118/) (2021)

## Built on by (most cited first, 10 of 10)

- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [SVeritas: Benchmark for Robust Speaker Verification under Diverse Conditions](https://mlsanity.com/paper/2509.17091/) (2025)
- [CN-Celeb-AV: A Multi-Genre Audio-Visual Dataset for Person Recognition](https://mlsanity.com/paper/2305.16049/) (2023)
- [CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing](https://mlsanity.com/paper/2412.04425/) (2024)
- [Effects of language mismatch in automatic forensic voice comparison using deep learning embeddings](https://mlsanity.com/paper/2209.12602/) (2022)
- [Adversarial Data Augmentation for Robust Speaker Verification](https://mlsanity.com/paper/2402.02699/) (2024)
- [Toroidal Probabilistic Spherical Discriminant Analysis](https://mlsanity.com/paper/2210.15441/) (2022)
- [Ordered and Binary Speaker Embedding](https://mlsanity.com/paper/2305.16043/) (2023)
- [Efficient Spoken Language Recognition via Multilabel Classification](https://mlsanity.com/paper/2306.01945/) (2023)
- [Efficient Multi-Model Fusion with Adversarial Complementary Representation Learning](https://mlsanity.com/paper/2404.15704/) (2024)

## Abstract (arXiv)

The 2021 Speaker Recognition Evaluation (SRE21) was the latest cycle of the ongoing evaluation series conducted by the U.S. National Institute of Standards and Technology (NIST) since 1996. It was the second large-scale multimodal speaker/person recognition evaluation organized by NIST (the first one being SRE19). Similar to SRE19, it featured two core evaluation tracks, namely audio and audio-visual, as well as an optional visual track. In addition to offering fixed and open training conditions, it also introduced new challenges for the community, thanks to a new multimodal (i.e., audio, video, and selfie images) and multilingual (i.e., with multilingual speakers) corpus, termed WeCanTalk, collected outside North America by the Linguistic Data Consortium (LDC). These challenges included: 1) trials (target and non-target) with enrollment and test segments originating from different domains (i.e., telephony versus video), and 2) trials (target and non-target) with enrollment and test segments spoken in different languages (i.e., cross-lingual trials). This paper presents an overview of SRE21 including the tasks, performance metric, data, evaluation protocol, results and system performance analyses. A total of 23 organizations (forming 15 teams) from academia and industry participated in SRE21 and submitted 158 valid system outputs. Evaluation results indicate: audio-visual fusion produce substantial gains in performance over audio-only or visual-only systems; top performing speaker and face recognition systems exhibited comparable performance under the matched domain conditions present in this evaluation; and, the use of complex neural network architectures (e.g., ResNet) along with angular losses with margin, data augmentation, as well as long duration fine-tuning contributed to notable performance improvements for the audio-only speaker recognition task.