# Out of a hundred trials, how many errors does your speaker verifier make?

arXiv: https://arxiv.org/abs/2104.00732 · HTML: https://mlsanity.com/paper/2104.00732/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 19 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 6 of 6)

- [The BOSARIS Toolkit: Theory, Algorithms and Code for Surviving the New DCF](https://mlsanity.com/paper/1304.2865/) (2013)
- [A Speaker Verification Backend with Robust Performance across Conditions](https://mlsanity.com/paper/2102.01760/) (2021)
- [The distribution of calibrated likelihood-ratios in speaker recognition](https://mlsanity.com/paper/1304.1199/) (2013)
- [A comparison of linear and non-linear calibrations for speaker recognition](https://mlsanity.com/paper/1402.2447/) (2014)
- [Generative Modelling for Unsupervised Score Calibration](https://mlsanity.com/paper/1311.0707/) (2013)
- [Bayesian calibration for forensic evidence reporting](https://mlsanity.com/paper/1403.5997/) (2014)

## Built on by (most cited first, 4 of 4)

- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [Analysis and Comparison of Classification Metrics](https://mlsanity.com/paper/2209.05355/) (2022)
- [t-EER: Parameter-Free Tandem Evaluation of Countermeasures and Biometric Comparators](https://mlsanity.com/paper/2309.12237/) (2023)
- [The Phonexia VoxCeleb Speaker Recognition Challenge 2021 System Description](https://mlsanity.com/paper/2109.02052/) (2021)

## Abstract (arXiv)

Out of a hundred trials, how many errors does your speaker verifier make? For the user this is an important, practical question, but researchers and vendors typically sidestep it and supply instead the conditional error-rates that are given by the ROC/DET curve. We posit that the user's question is answered by the Bayes error-rate. We present a tutorial to show how to compute the error-rate that results when making Bayes decisions with calibrated likelihood ratios, supplied by the verifier, and an hypothesis prior, supplied by the user. For perfect calibration, the Bayes error-rate is upper bounded by min(EER,P,1-P), where EER is the equal-error-rate and P, 1-P are the prior probabilities of the competing hypotheses. The EER represents the accuracy of the verifier, while min(P,1-P) represents the hardness of the classification problem. We further show how the Bayes error-rate can be computed also for non-perfect calibration and how to generalize from error-rate to expected cost. We offer some criticism of decisions made by direct score thresholding. Finally, we demonstrate by analyzing error-rates of the recently published DCA-PLDA speaker verifier.