# Model-Based Approach for Measuring the Fairness in ASR

arXiv: https://arxiv.org/abs/2109.09061 · HTML: https://mlsanity.com/paper/2109.09061/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 29 references that are in the map.
- First posted on arXiv 19 Sep 2021.

## Builds on (most cited first, 7 of 7)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://mlsanity.com/paper/1608.08868/) (2016)
- [Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition](https://mlsanity.com/paper/2010.10759/) (2020)
- [Investigating Bias and Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2007.10075/) (2020)

## Built on by (most cited first, 13 of 13)

- [Towards measuring fairness in speech recognition: Fair-Speech dataset](https://mlsanity.com/paper/2408.12734/) (2024)
- [Hey ASR System! Why Aren't You More Inclusive? Automatic Speech Recognition Systems' Bias and Proposed Bias Mitigation Techniques. A Literature Review](https://mlsanity.com/paper/2211.09511/) (2022)
- [A Study of Gender Impact in Self-supervised Models for Speech-to-Text Systems](https://mlsanity.com/paper/2204.01397/) (2022)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognition](https://mlsanity.com/paper/2310.18877/) (2023)
- [A Contrastive Learning Approach to Mitigate Bias in Speech Models](https://mlsanity.com/paper/2406.14686/) (2024)
- [Global Performance Disparities Between English-Language Accents in Automatic Speech Recognition](https://mlsanity.com/paper/2208.01157/) (2022)
- [Sonos Voice Control Bias Assessment Dataset: A Methodology for Demographic Bias Assessment in Voice Assistants](https://mlsanity.com/paper/2405.19342/) (2024)
- [How to Evaluate Automatic Speech Recognition: Comparing Different Performance and Bias Measures](https://mlsanity.com/paper/2507.05885/) (2025)
- [Everyone deserves their voice to be heard: Analyzing Predictive Gender Bias in ASR Models Applied to Dutch Speech Data](https://mlsanity.com/paper/2411.09431/) (2024)
- [Towards Fair ASR For Second Language Speakers Using Fairness Prompted Finetuning](https://mlsanity.com/paper/2510.18374/) (2025)
- [Measuring Fairness in Large Audio Language Models via Semantic-Aware Bias Estimation](https://mlsanity.com/paper/2608.13624/) (2026)
- [A Cross Community Agenda for Speech AI](https://mlsanity.com/paper/2609.13168/) (2026)

## Abstract (arXiv)

The issue of fairness arises when the automatic speech recognition (ASR) systems do not perform equally well for all subgroups of the population. In any fairness measurement studies for ASR, the open questions of how to control the nuisance factors, how to handle unobserved heterogeneity across speakers, and how to trace the source of any word error rate (WER) gap among different subgroups are especially important - if not appropriately accounted for, incorrect conclusions will be drawn. In this paper, we introduce mixed-effects Poisson regression to better measure and interpret any WER difference among subgroups of interest. Particularly, the presented method can effectively address the three problems raised above and is very flexible to use in practical disparity analyses. We demonstrate the validity of proposed model-based approach on both synthetic and real-world speech data.