mlsanityData to 14 Oct 2026

Fairness Beyond a Single Run: Training-Seed Variability in Speech LLM Adaptation

arXiv 2609.38976 ↗First posted 30 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Demographic fairness gaps in automatic speech recognition are almost always reported from a single training run. We fine-tune the Q-former projector and LoRA adapters of a speech LLM at five audio compression factors and six random seeds, holding the encoder, base decoder, data and decoding fixed, and evaluate every run on Common Voice and Fair-Speech. At 460 h of clean LibriSpeech, the seed moves fairness metrics more than compression does on most demographic axes. A balanced 3x3 decomposition attributes 85.3% of the variation in Fair-Speech ethnicity normalized gap to the seed against 8.3% to compression (p = 0.009), though compression explains more on age and gender. Held-out LibriSpeech word error rate spreads by 0.04 points across those seeds while Common Voice spreads by 8.57, so these are not failed runs, and the effect survives controlling for accuracy and dropout. Scaling and diversifying the adaptation set to 960 h damps the effect but does not remove it. On Fair-Speech ethnicity, two single-run systems must differ by more than 0.30 in normalized gap to exceed seed variability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 23 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

17 of 23 references.

References · 17

  1. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  2. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
  3. 2021Your fairness may vary: Pretrained language model fairness in toxic text classificationcited by 39
  4. 2025Granite-speech: open-source speech-aware LLMs with strong English ASR capabilitiescited by 27
  5. 2023On The Impact of Machine Learning Randomness on Group Fairnesscited by 26
  6. 2024Towards measuring fairness in speech recognition: Fair-Speech datasetcited by 26
  7. 2023Svarah: Evaluating English ASR Systems on Indian Accentscited by 16
  8. 2024Lost in Transcription: Identifying and Quantifying the Accuracy Biases of Automatic Speech Recognition Systems Against Disfluent Speechcited by 15
  9. 2025Automatic Speech Recognition Biases in Newcastle English: an Error Analysiscited by 6
  10. 2025ASR-FAIRBENCH: Measuring and Benchmarking Equity Across Speech Recognition Systemscited by 4
  11. 2026Bias in the Ear of the Listener: Assessing Sensitivity in Audio Language Models Across Linguistic, Demographic, and Positional Variationscited by 4
  12. 2026Do LLM Decoders Listen Fairly? Benchmarking How Language Model Priors Shape Bias in Speech Recognitioncited by 4
5 more
  1. 2025FairASR: Fair Audio Contrastive Learning for Automatic Speech Recognitioncited by 3
  2. 2026Toward Fair Speech Technologies: A Comprehensive Survey of Bias and Fairness in Speech AIcited by 3
  3. 2026The Voice of Equity: A Systematic Evaluation of Bias Mitigation Techniques for Speech-Based Cognitive Impairment Detection Across Architectures and Demographicscited by 2
  4. 2026Responsible Benchmarking of Fairness for Automatic Speech Recognitioncited by 1
  5. 2026Seeds Before Objectives: Rethinking Evaluation for Low-Resource Garhwali ASRcited by 1

Built on by

Not cited yet.