mlsanityData to 7 Oct 2026

Towards measuring fairness in speech recognition: Fair-Speech dataset

arXiv 2408.12734 ↗First posted 22 Aug 2024cs.AI cs.CY cs.SDcited by 26

Not on the map. Why?

Abstract (arXiv)

The current public datasets for speech recognition (ASR) tend not to focus specifically on the fairness aspect, such as performance across different demographic groups. This paper introduces a novel dataset, Fair-Speech, a publicly released corpus to help researchers evaluate their ASR models for accuracy across a diverse set of self-reported demographic information, such as age, gender, ethnicity, geographic variation and whether the participants consider themselves native English speakers. Our dataset includes approximately 26.5K utterances in recorded speech by 593 people in the United States, who were paid to record and submit audios of themselves saying voice commands. We also provide ASR baselines, including on models trained on transcribed and untranscribed social media videos and open source models.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 9 of its 23 references that are in the map. First posted on arXiv 22 Aug 2024.

Builds on

9 of 23 references.

References · 9

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  3. 2017Counterfactual Fairnesscited by 802
  4. 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 111
  5. 2022Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparitiescited by 27
  6. 2021Towards Measuring Fairness in Speech Recognition: Casual Conversations Dataset Transcriptionscited by 26
  7. 2021ASR4REAL: An extended benchmark for speech modelscited by 14
  8. 2021Model-Based Approach for Measuring the Fairness in ASRcited by 13
  9. 2023Improving Fairness and Robustness in End-to-End Speech Recognition through unsupervised clusteringcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 13; 2026: 13.

26 papers.

Sort:

By area:

  1. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 47
  2. 2025Audio Large Language Models Can Be Descriptive Speech Quality Evaluatorscited by 25
  3. 2025CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speechcited by 11
  4. 2025Mind the Gap! Static and Interactive Evaluations of Large Audio Modelscited by 8
  5. 2025Voxlect: A Speech Foundation Model Benchmark for Modeling Dialects and Regional Languages Around the Globecited by 5
  6. 2025Evaluating Bias in Spoken Dialogue LLMs for Real-World Decisions and Recommendationscited by 5
  7. 2025The ML-SUPERB 2.0 Challenge: Towards Inclusive ASR Benchmarking for All Language Varietiescited by 4
  8. 2025ASR-FAIRBENCH: Measuring and Benchmarking Equity Across Speech Recognition Systemscited by 3
  9. 2025WavShape: Information-Theoretic Speech Representation Learning for Fair and Privacy-Aware Audio Processingcited by 2
  10. 2026DIFFA-2: A Practical Diffusion Large Language Model for General Audio Understandingcited by 2
  11. 2025Fairness in Dysarthric Speech Synthesis: Understanding Intrinsic Bias in Dysarthric Speech Cloning using F5-TTScited by 1
  12. 2025Acoustic-based Gender Differentiation in Speech-aware Language Modelscited by 1
14 more
  1. 2026Beyond Word Error Rate: Auditing the Diversity Tax in Speech Recognition through Dataset Cartographycited by 1
  2. 2026Where Do Self-Supervised Speech Models Become Unfair?cited by 1
  3. 2026Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition modelscited by 1
  4. 2025Towards Fair ASR For Second Language Speakers Using Fairness Prompted Finetuningcited by 0
  5. 2025MedVoiceBias: A Controlled Study of Audio LLM Behavior in Clinical Decision-Makingcited by 0
  6. 2026"OK Aura, Be Fair With Me": Demographics-Agnostic Training for Bias Mitigation in Wake-up Word Detectioncited by 0
  7. 2026Cross-Cultural Bias in Mel-Scale Representations: Evidence and Alternatives from Speech and Musiccited by 0
  8. 2026Premover: Fast Vision-Language-Action Control by Acting Before Instructions Are Completecited by 0
  9. 2026Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Textcited by 0
  10. 2026Linguistic Distance Segregates Latent Representations in Automatic Speech Recognition Systemscited by 0
  11. 2026Queer inclusion in speech datasets: An audit and taxonomy of practical tensionscited by 0
  12. 2026Towards participatory speech dataset curation: A queer case study and conceptual frameworkcited by 0
  13. 2026When Entanglement Lower-Bounds Disparity: Auditing and Repairing Demographic Fairness in Audio Understanding Modelscited by 0
  14. 2026Fairness Beyond a Single Run: Training-Seed Variability in Speech LLM Adaptationcited by 0