# The BOSARIS Toolkit: Theory, Algorithms and Code for Surviving the New DCF

arXiv: https://arxiv.org/abs/1304.2865 · HTML: https://mlsanity.com/paper/1304.2865/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2013.

## Built on by (most cited first, 12 of 12)

- [Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detection](https://mlsanity.com/paper/2406.17376/) (2024)
- [Exploring WavLM Back-ends for Speech Spoofing and Deepfake Detection](https://mlsanity.com/paper/2409.05032/) (2024)
- [Forensic deepfake audio detection using segmental speech features](https://mlsanity.com/paper/2505.13847/) (2025)
- [Learnable MFCCs for Speaker Verification](https://mlsanity.com/paper/2102.10322/) (2021)
- [STC speaker recognition systems for the NIST SRE 2021](https://mlsanity.com/paper/2111.02298/) (2021)
- [a-DCF: an architecture agnostic metric with application to spoofing-robust speaker verification](https://mlsanity.com/paper/2403.01355/) (2024)
- [Out of a hundred trials, how many errors does your speaker verifier make?](https://mlsanity.com/paper/2104.00732/) (2021)
- [CN-Celeb-AV: A Multi-Genre Audio-Visual Dataset for Person Recognition](https://mlsanity.com/paper/2305.16049/) (2023)
- [t-EER: Parameter-Free Tandem Evaluation of Countermeasures and Biometric Comparators](https://mlsanity.com/paper/2309.12237/) (2023)
- [LMD: A Learnable Mask Network to Detect Adversarial Examples for Speaker Verification](https://mlsanity.com/paper/2211.00825/) (2022)
- [Improving Presentation Attack Detection for ID Cards on Remote Verification Systems](https://mlsanity.com/paper/2301.09542/) (2023)
- [Optimizing Multi-Taper Features for Deep Speaker Verification](https://mlsanity.com/paper/2110.10983/) (2021)

## Abstract (arXiv)

The change of two orders of magnitude in the 'new DCF' of NIST's SRE'10, relative to the 'old DCF' evaluation criterion, posed a difficult challenge for participants and evaluator alike. Initially, participants were at a loss as to how to calibrate their systems, while the evaluator underestimated the required number of evaluation trials. After the fact, it is now obvious that both calibration and evaluation require very large sets of trials. This poses the challenges of (i) how to decide what number of trials is enough, and (ii) how to process such large data sets with reasonable memory and CPU requirements. After SRE'10, at the BOSARIS Workshop, we built solutions to these problems into the freely available BOSARIS Toolkit. This paper explains the principles and algorithms behind this toolkit. The main contributions of the toolkit are: 1. The Normalized Bayes Error-Rate Plot, which analyses likelihood- ratio calibration over a wide range of DCF operating points. These plots also help in judging the adequacy of the sizes of calibration and evaluation databases. 2. Efficient algorithms to compute DCF and minDCF for large score files, over the range of operating points required by these plots. 3. A new score file format, which facilitates working with very large trial lists. 4. A faster logistic regression optimizer for fusion and calibration. 5. A principled way to define EER (equal error rate), which is of practical interest when the absolute error count is small.