# Fairness Beyond a Single Run: Training-Seed Variability in Speech LLM Adaptation

arXiv: https://arxiv.org/abs/2609.38976 · HTML: https://mlsanity.com/paper/2609.38976/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 17 of 17)

- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Your fairness may vary: Pretrained language model fairness in toxic text classification](https://mlsanity.com/paper/2108.01250/) (2021)
- [Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities](https://mlsanity.com/paper/2505.08699/) (2025)
- [On The Impact of Machine Learning Randomness on Group Fairness](https://mlsanity.com/paper/2307.04138/) (2023)
- [Towards measuring fairness in speech recognition: Fair-Speech dataset](https://mlsanity.com/paper/2408.12734/) (2024)
- [Svarah: Evaluating English ASR Systems on Indian Accents](https://mlsanity.com/paper/2305.15760/) (2023)
- [Lost in Transcription: Identifying and Quantifying the Accuracy Biases of Automatic Speech Recognition Systems Against Disfluent Speech](https://mlsanity.com/paper/2405.06150/) (2024)
- [Automatic Speech Recognition Biases in Newcastle English: an Error Analysis](https://mlsanity.com/paper/2506.16558/) (2025)
- [ASR-FAIRBENCH: Measuring and Benchmarking Equity Across Speech Recognition Systems](https://mlsanity.com/paper/2505.11572/) (2025)
- [Bias in the Ear of the Listener: Assessing Sensitivity in Audio Language Models Across Linguistic, Demographic, and Positional Variations](https://mlsanity.com/paper/2602.01030/) (2026)
- [Do LLM Decoders Listen Fairly? Benchmarking How Language Model Priors Shape Bias in Speech Recognition](https://mlsanity.com/paper/2604.21276/) (2026)
- [FairASR: Fair Audio Contrastive Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2506.10747/) (2025)
- [Toward Fair Speech Technologies: A Comprehensive Survey of Bias and Fairness in Speech AI](https://mlsanity.com/paper/2605.01597/) (2026)
- [The Voice of Equity: A Systematic Evaluation of Bias Mitigation Techniques for Speech-Based Cognitive Impairment Detection Across Architectures and Demographics](https://mlsanity.com/paper/2601.16989/) (2026)
- [Responsible Benchmarking of Fairness for Automatic Speech Recognition](https://mlsanity.com/paper/2605.10615/) (2026)
- [Seeds Before Objectives: Rethinking Evaluation for Low-Resource Garhwali ASR](https://mlsanity.com/paper/2608.10670/) (2026)

## Abstract (arXiv)

Demographic fairness gaps in automatic speech recognition are almost always reported from a single training run. We fine-tune the Q-former projector and LoRA adapters of a speech LLM at five audio compression factors and six random seeds, holding the encoder, base decoder, data and decoding fixed, and evaluate every run on Common Voice and Fair-Speech. At 460 h of clean LibriSpeech, the seed moves fairness metrics more than compression does on most demographic axes. A balanced 3x3 decomposition attributes 85.3% of the variation in Fair-Speech ethnicity normalized gap to the seed against 8.3% to compression (p = 0.009), though compression explains more on age and gender. Held-out LibriSpeech word error rate spreads by 0.04 points across those seeds while Common Voice spreads by 8.57, so these are not failed runs, and the effect survives controlling for accuracy and dropout. Scaling and diversifying the adaptation set to 960 h damps the effect but does not remove it. On Fair-Speech ethnicity, two single-run systems must differ by more than 0.30 in normalized gap to exceed seed variability.