# Confidence Intervals for Error Rates in 1:1 Matching Tasks: Critical Statistical Analysis and Recommendations

arXiv: https://arxiv.org/abs/2306.01198 · HTML: https://mlsanity.com/paper/2306.01198/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 66 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Benchmarking Representation Learning for Natural World Image Collections](https://mlsanity.com/paper/2103.16483/) (2021)
- [Towards causal benchmarking of bias in face analysis algorithms](https://mlsanity.com/paper/2007.06570/) (2020)
- [Characterizing the Variability in Face Recognition Accuracy Relative to Race](https://mlsanity.com/paper/1904.07325/) (2019)
- [Network Data](https://mlsanity.com/paper/1912.06346/) (2019)
- [Subsampling bootstrap of count features of networks](https://mlsanity.com/paper/1312.2645/) (2013)
- [Additive and multiplicative effects network models](https://mlsanity.com/paper/1807.08038/) (2018)
- [Bootstrapping Exchangeable Random Graphs](https://mlsanity.com/paper/1711.00813/) (2017)
- [Cluster-Robust Variance Estimation for Dyadic Data](https://mlsanity.com/paper/1312.3398/) (2013)

## Built on by (most cited first, 1 of 1)

- [A Rapid Test for Accuracy and Bias of Face Recognition Technology](https://mlsanity.com/paper/2502.14996/) (2025)

## Abstract (arXiv)

Matching algorithms are commonly used to predict matches between items in a collection. For example, in 1:1 face verification, a matching algorithm predicts whether two face images depict the same person. Accurately assessing the uncertainty of the error rates of such algorithms can be challenging when data are dependent and error rates are low, two aspects that have been often overlooked in the literature. In this work, we review methods for constructing confidence intervals for error rates in 1:1 matching tasks. We derive and examine the statistical properties of these methods, demonstrating how coverage and interval width vary with sample size, error rates, and degree of data dependence on both analysis and experiments with synthetic and real-world datasets. Based on our findings, we provide recommendations for best practices for constructing confidence intervals for error rates in 1:1 matching tasks.