# To train or not to train adversarially: A study of bias mitigation strategies for speaker recognition

arXiv: https://arxiv.org/abs/2203.09122 · HTML: https://mlsanity.com/paper/2203.09122/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Mar 2022.

## Built on by (most cited first, 8 of 8)

- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [Shortcut Learning in Binary Classifier Black Boxes: Applications to Voice Anti-Spoofing and Biometrics](https://mlsanity.com/paper/2601.17782/) (2026)
- [Causally Disentangled Contrastive Learning for Multilingual Speaker Embeddings](https://mlsanity.com/paper/2602.01363/) (2026)
- [Where Do Self-Supervised Speech Models Become Unfair?](https://mlsanity.com/paper/2604.18249/) (2026)
- [What You Train Is What You Get: Gender Bias, Training Composition, and Post-Hoc Mitigation in Audio Deepfake Detection](https://mlsanity.com/paper/2607.09891/) (2026)
- [Simple Language Normalization Wins: Cross-Lingual Speaker Verification for the TidyVoice 2026 Challenge](https://mlsanity.com/paper/2607.22923/) (2026)
- [The Voiceprint Fallacy: Why Voices Are Not Unique Biometric Imprints](https://mlsanity.com/paper/2608.07980/) (2026)
- [InterBias-SV: Compound Conditions in Speaker Verification](https://mlsanity.com/paper/2609.36500/) (2026)

## Abstract (arXiv)

Speaker recognition is increasingly used in several everyday applications including smart speakers, customer care centers and other speech-driven analytics. It is crucial to accurately evaluate and mitigate biases present in machine learning (ML) based speech technologies, such as speaker recognition, to ensure their inclusive adoption. ML fairness studies with respect to various demographic factors in modern speaker recognition systems are lagging compared to other human-centered applications such as face recognition. Existing studies on fairness in speaker recognition systems are largely limited to evaluating biases at specific operating points of the systems, which can lead to false expectations of fairness. Moreover, there are only a handful of bias mitigation strategies developed for speaker recognition systems. In this paper, we systematically evaluate the biases present in speaker recognition systems with respect to gender across a range of system operating points. We also propose adversarial and multi-task learning techniques to improve the fairness of these systems. We show through quantitative and qualitative evaluations that the proposed methods improve the fairness of ASV systems over baseline methods trained using data balancing techniques. We also present a fairness-utility trade-off analysis to jointly examine fairness and the overall system performance. We show that although systems trained using adversarial techniques improve fairness, they are prone to reduced utility. On the other hand, multi-task methods can improve the fairness while retaining the utility. These findings can inform the choice of bias mitigation strategies in the field of speaker recognition.