# Sum of Group Error Differences: A Critical Examination of Bias Evaluation in Biometric Verification and a Dual-Metric Measure

arXiv: https://arxiv.org/abs/2404.15385 · HTML: https://mlsanity.com/paper/2404.15385/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 23 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning](https://mlsanity.com/paper/1911.10692/) (2019)
- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [Face Recognition: Too Bias, or Not Too Bias?](https://mlsanity.com/paper/2002.06483/) (2020)
- [SensitiveLoss: Improving Accuracy and Fairness of Face Representations with Discrimination-Aware Deep Learning](https://mlsanity.com/paper/2004.11246/) (2020)
- [Fairness in Biometrics: a figure of merit to assess biometric verification systems](https://mlsanity.com/paper/2011.02395/) (2020)
- [Characterizing the Variability in Face Recognition Accuracy Relative to Race](https://mlsanity.com/paper/1904.07325/) (2019)
- [An Experimental Evaluation of Covariates Effects on Unconstrained Face Verification](https://mlsanity.com/paper/1808.05508/) (2018)
- [Evaluating Proposed Fairness Models for Face Recognition Algorithms](https://mlsanity.com/paper/2203.05051/) (2022)
- [DocFace: Matching ID Document Photos to Selfies](https://mlsanity.com/paper/1805.02283/) (2018)

## Built on by (most cited first, 1 of 1)

- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)

## Abstract (arXiv)

Biometric Verification (BV) systems often exhibit accuracy disparities across different demographic groups, leading to biases in BV applications. Assessing and quantifying these biases is essential for ensuring the fairness of BV systems. However, existing bias evaluation metrics in BV have limitations, such as focusing exclusively on match or non-match error rates, overlooking bias on demographic groups with performance levels falling between the best and worst performance levels, and neglecting the magnitude of the bias present. This paper presents an in-depth analysis of the limitations of current bias evaluation metrics in BV and, through experimental analysis, demonstrates their contextual suitability, merits, and limitations. Additionally, it introduces a novel general-purpose bias evaluation measure for BV, the ``Sum of Group Error Differences (SEDG)''. Our experimental results on controlled synthetic datasets demonstrate the effectiveness of demographic bias quantification when using existing metrics and our own proposed measure. We discuss the applicability of the bias evaluation metrics in a set of simulated demographic bias scenarios and provide scenario-based metric recommendations. Our code is publicly available under urlhttps://github.com/alaaobeid/SEDG.