# ASR-FAIRBENCH: Measuring and Benchmarking Equity Across Speech Recognition Systems

arXiv: https://arxiv.org/abs/2505.11572 · HTML: https://mlsanity.com/paper/2505.11572/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 6 references that are in the map.
- First posted on arXiv 16 May 2025.

## Builds on (most cited first, 3 of 3)

- [Towards measuring fairness in speech recognition: Fair-Speech dataset](https://mlsanity.com/paper/2408.12734/) (2024)
- [A Deep Dive into the Disparity of Word Error Rates Across Thousands of NPTEL MOOC Videos](https://mlsanity.com/paper/2307.10587/) (2023)
- [Open Universal Arabic ASR Leaderboard](https://mlsanity.com/paper/2412.13788/) (2024)

## Built on by (most cited first, 4 of 4)

- [VoxSafeBench: Not Just What Is Said, but Who, How, and Where](https://mlsanity.com/paper/2604.14548/) (2026)
- [Where Do Self-Supervised Speech Models Become Unfair?](https://mlsanity.com/paper/2604.18249/) (2026)
- [ACES: Accent Subspaces for Coupling, Explanations, and Stress-Testing in Automatic Speech Recognition](https://mlsanity.com/paper/2603.03359/) (2026)
- [Fairness Beyond a Single Run: Training-Seed Variability in Speech LLM Adaptation](https://mlsanity.com/paper/2609.38976/) (2026)

## Abstract (arXiv)

Automatic Speech Recognition (ASR) systems have become ubiquitous in everyday applications, yet significant disparities in performance across diverse demographic groups persist. In this work, we introduce the ASR-FAIRBENCH leaderboard which is designed to assess both the accuracy and equity of ASR models in real-time. Leveraging the Meta's Fair-Speech dataset, which captures diverse demographic characteristics, we employ a mixed-effects Poisson regression model to derive an overall fairness score. This score is integrated with traditional metrics like Word Error Rate (WER) to compute the Fairness Adjusted ASR Score (FAAS), providing a comprehensive evaluation framework. Our approach reveals significant performance disparities in SOTA ASR models across demographic groups and offers a benchmark to drive the development of more inclusive ASR technologies.