# A Speaker Verification Backend with Robust Performance across Conditions

arXiv: https://arxiv.org/abs/2102.01760 · HTML: https://mlsanity.com/paper/2102.01760/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 68 references that are in the map.
- First posted on arXiv 2 Feb 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Voices Obscured in Complex Environmental Settings (VOICES) corpus](https://mlsanity.com/paper/1804.05053/) (2018)
- [The PAV algorithm optimizes binary proper scoring rules](https://mlsanity.com/paper/1304.2331/) (2013)
- [The distribution of calibrated likelihood-ratios in speaker recognition](https://mlsanity.com/paper/1304.1199/) (2013)
- [Likelihood-ratio calibration using prior-weighted proper scoring rules](https://mlsanity.com/paper/1307.7981/) (2013)
- [A comparison of linear and non-linear calibrations for speaker recognition](https://mlsanity.com/paper/1402.2447/) (2014)
- [End-to-end DNN Based Speaker Recognition Inspired by i-vector and PLDA](https://mlsanity.com/paper/1710.02369/) (2017)
- [Fast variational Bayes for heavy-tailed PLDA applied to i-vectors and x-vectors](https://mlsanity.com/paper/1803.09153/) (2018)
- [A discriminative condition-aware backend for speaker verification](https://mlsanity.com/paper/1911.11622/) (2019)
- [A Speaker Verification Backend for Improved Calibration Performance across Varying Conditions](https://mlsanity.com/paper/2002.03802/) (2020)
- [Neural PLDA Modeling for End-to-End Speaker Verification](https://mlsanity.com/paper/2008.04527/) (2020)

## Built on by (most cited first, 10 of 10)

- [Evaluating Posterior Probabilities: Decision Theory, Proper Scoring Rules, and Calibration](https://mlsanity.com/paper/2408.02841/) (2024)
- [QFA2SR: Query-Free Adversarial Transfer Attacks to Speaker Recognition Systems](https://mlsanity.com/paper/2305.14097/) (2023)
- [Out of a hundred trials, how many errors does your speaker verifier make?](https://mlsanity.com/paper/2104.00732/) (2021)
- [t-EER: Parameter-Free Tandem Evaluation of Countermeasures and Biometric Comparators](https://mlsanity.com/paper/2309.12237/) (2023)
- [A Discriminative Hierarchical PLDA-based Model for Spoken Language Recognition](https://mlsanity.com/paper/2201.01364/) (2022)
- [Optimizing Tandem Speaker Verification and Anti-Spoofing Systems](https://mlsanity.com/paper/2201.09709/) (2022)
- [Improved Relation Networks for End-to-End Speaker Verification and Identification](https://mlsanity.com/paper/2203.17218/) (2022)
- [Meta-Learning Framework for End-to-End Imposter Identification in Unseen Speaker Recognition](https://mlsanity.com/paper/2306.00952/) (2023)
- [Cosine Scoring with Uncertainty for Neural Speaker Embedding](https://mlsanity.com/paper/2403.06404/) (2024)
- [Speech Generation Speaker Poisoning: Capability Erasure in Zero-Shot Text-to-Speech](https://mlsanity.com/paper/2603.07551/) (2026)

## Abstract (arXiv)

In this paper, we address the problem of speaker verification in conditions unseen or unknown during development. A standard method for speaker verification consists of extracting speaker embeddings with a deep neural network and processing them through a backend composed of probabilistic linear discriminant analysis (PLDA) and global logistic regression score calibration. This method is known to result in systems that work poorly on conditions different from those used to train the calibration model. We propose to modify the standard backend, introducing an adaptive calibrator that uses duration and other automatically extracted side-information to adapt to the conditions of the inputs. The backend is trained discriminatively to optimize binary cross-entropy. When trained on a number of diverse datasets that are labeled only with respect to speaker, the proposed backend consistently and, in some cases, dramatically improves calibration, compared to the standard PLDA approach, on a number of held-out datasets, some of which are markedly different from the training data. Discrimination performance is also consistently improved. We show that joint training of the PLDA and the adaptive calibrator is essential -- the same benefits cannot be achieved when freezing PLDA and fine-tuning the calibrator. To our knowledge, the results in this paper are the first evidence in the literature that it is possible to develop a speaker verification system with robust out-of-the-box performance on a large variety of conditions.