mlsanityData to 7 Oct 2026

Where Do Self-Supervised Speech Models Become Unfair?

arXiv 2604.18249 ↗First posted 20 Apr 2026cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Speech encoder models are known to model members of some speaker groups (SGs) better than others. However, there has been little work in establishing why this occurs on a technological level. To our knowledge, we present the first layerwise fairness analysis of pretrained self-supervised speech encoder models (S3Ms), probing each embedding layer for speaker identification (SID) automatic speech recognition (ASR). We find S3Ms produce embeddings biased against certain SGs for both tasks, starting at the very first latent layers. Furthermore, we find opposite patterns of layerwise bias for SID vs ASR for all models in our study: SID bias is minimized in layers that minimize overall SID error; on the other hand, ASR bias is maximized in layers that minimize overall ASR error. The inverse bias/error relationship for ASR is unaffected when probing S3Ms that are finetuned for ASR, suggesting SG-level bias is established during pretraining and is difficult to remove.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

30 of 46 references.

References · 30

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  7. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  8. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  9. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  10. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 158
  11. 2022Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparitiescited by 27
  12. 2024Towards measuring fairness in speech recognition: Fair-Speech datasetcited by 26
18 more
  1. 2024Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gapscited by 24
  2. 2019Gender Representation in French Broadcast Corpora and Its Impact on ASR Performancecited by 23
  3. 2021Don't speak too fast: The impact of data bias on self-supervised speech modelscited by 20
  4. 2022Bias in Automated Speaker Recognitioncited by 20
  5. 2021Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learningcited by 17
  6. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 17
  7. 2021Accent-Robust Automatic Speech Recognition Using Supervised and Unsupervised Wav2vec Embeddingscited by 12
  8. 2023Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognitioncited by 9
  9. 2024Open Implementation and Study of BEST-RQ for Speech Processingcited by 9
  10. 2022To train or not to train adversarially: A study of bias mitigation strategies for speaker recognitioncited by 8
  11. 2022Enhancing and Adversarial: Improve ASR with Speaker Labelscited by 6
  12. 2024Sonos Voice Control Bias Assessment Dataset: A Methodology for Demographic Bias Assessment in Voice Assistantscited by 6
  13. 2022Joint Encoder-Decoder Self-Supervised Pre-training for ASRcited by 4
  14. 2025ASR-FAIRBENCH: Measuring and Benchmarking Equity Across Speech Recognition Systemscited by 3
  15. 2025FairASR: Fair Audio Contrastive Learning for Automatic Speech Recognitioncited by 3
  16. 2023Some voices are too common: Building fair speech recognition systems using the Common Voice datasetcited by 2
  17. 2025Exploring Gender Disparities in Automatic Speech Recognition Technologycited by 2
  18. 2025Improving ASR Fairness for Cleft Lip and Palate Speech: A Study on Severity-Aware Data Mixingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition modelscited by 1