# A Comparison of Differential Performance Metrics for the Evaluation of Automatic Speaker Verification Fairness

arXiv: https://arxiv.org/abs/2404.17810 · HTML: https://mlsanity.com/paper/2404.17810/

## Key facts

- Cited by 3 papers in the map (data through 19 Oct 2026).
- First posted on arXiv 27 Apr 2024.

## Built on by (most cited first, 3 of 3)

- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [When Fine-Tuning is Not Enough: Lessons from HSAD on Hybrid and Adversarial Audio Spoof Detection](https://mlsanity.com/paper/2509.07323/) (2025)
- [VoxAnon: A balanced dataset for a fair evaluation of speech anonymization](https://mlsanity.com/paper/2610.13313/) (2026)

## Abstract (arXiv)

When decisions are made and when personal data is treated by automated processes, there is an expectation of fairness -- that members of different demographic groups receive equitable treatment. This expectation applies to biometric systems such as automatic speaker verification (ASV). We present a comparison of three candidate fairness metrics and extend previous work performed for face recognition, by examining differential performance across a range of different ASV operating points. Results show that the Gini Aggregation Rate for Biometric Equitability (GARBE) is the only one which meets three functional fairness measure criteria. Furthermore, a comprehensive evaluation of the fairness and verification performance of five state-of-the-art ASV systems is also presented. Our findings reveal a nuanced trade-off between fairness and verification accuracy underscoring the complex interplay between system design, demographic inclusiveness, and verification reliability.