mlsanityData to 7 Oct 2026

On the social bias of speech self-supervised models

arXiv 2406.04997 ↗First posted 7 Jun 2024eess.AS cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) speech models have achieved remarkable performance in various tasks, yet the biased outcomes, especially affecting marginalized groups, raise significant concerns. Social bias refers to the phenomenon where algorithms potentially amplify disparate properties between social groups present in the data used for training. Bias in SSL models can perpetuate injustice by automating discriminatory patterns and reinforcing inequitable systems. This work reveals that prevalent SSL models inadvertently acquire biased associations. We probe how various factors, such as model architecture, size, and training methodologies, influence the propagation of social bias within these models. Finally, we explore the efficacy of debiasing SSL models through regularization techniques, specifically via model compression. Our findings reveal that employing techniques such as row-pruning and training wider, shallower models can effectively mitigate social bias within SSL model.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

24 of 33 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  7. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  8. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  9. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 196
  10. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
  11. 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
  12. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 129
12 more
  1. 2020Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biasescited by 107
  2. 2020Unsupervised pretraining transfers well across languagescited by 100
  3. 2023Fast Model Debias with Machine Unlearningcited by 70
  4. 2022Speech Emotion Recognition using Self-Supervised Featurescited by 52
  5. 2022Do self-supervised speech models develop human-like perception biases?cited by 28
  6. 2022Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparitiescited by 27
  7. 2021Don't speak too fast: The impact of data bias on self-supervised speech modelscited by 20
  8. 2023Understanding the Effect of Model Compression on Social Bias in Large Language Modelscited by 19
  9. 2022Can Model Compression Improve NLP Fairnesscited by 18
  10. 2022A Study of Gender Impact in Self-supervised Models for Speech-to-Text Systemscited by 14
  11. 2023Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognitioncited by 9
  12. 2024The Effect of Batch Size on Contrastive Self-Supervised Speech Representation Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 7.

16 papers.

Sort:

By area:

  1. 2024Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Modelscited by 20
  2. 2024Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Modelscited by 15
  3. 2024Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budgetcited by 5
  4. 2025Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approachcited by 5
  5. 2025ToxicTone: A Mandarin Audio Dataset Annotated for Toxicity and Toxic Utterance Tonalitycited by 5
  6. 2025Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translationcited by 5
  7. 2025EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognitioncited by 3
  8. 2026AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognitioncited by 2
  9. 2025CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognitioncited by 1
  10. 2025Beyond Bias Scores: Unmasking Vacuous Neutrality in Small Language Modelscited by 1
  11. 2026Bias and Fairness in Self-Supervised Acoustic Representations for Cognitive Impairment Detectioncited by 1
  12. 2026The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluationcited by 1
4 more
  1. 2026"OK Aura, Be Fair With Me": Demographics-Agnostic Training for Bias Mitigation in Wake-up Word Detectioncited by 0
  2. 2026Fair Cognitive Impairment Detection Through Unlearningcited by 0
  3. 2026A Probe Shift Is Not a Fairness Fix: The Limits of Representation Steering in Speech Modelscited by 0
  4. 2026Pruning for Efficiency, Paying in Fairness: Demographic Disparities in Pruned Speech-LLMscited by 0