# A Study of Gender Impact in Self-supervised Models for Speech-to-Text Systems

arXiv: https://arxiv.org/abs/2204.01397 · HTML: https://mlsanity.com/paper/2204.01397/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Layer-wise Analysis of a Self-supervised Speech Representation Model](https://mlsanity.com/paper/2107.04734/) (2021)
- [fairseq S2T: Fast Speech-to-Text Modeling with fairseq](https://mlsanity.com/paper/2010.05171/) (2020)
- [The Multilingual TEDx Corpus for Speech Recognition and Translation](https://mlsanity.com/paper/2102.01757/) (2021)
- [ESPnet-ST: All-in-One Speech Translation Toolkit](https://mlsanity.com/paper/2004.10234/) (2020)
- [Effectiveness of self-supervised pre-training for speech recognition](https://mlsanity.com/paper/1911.03912/) (2019)
- [Quantifying Bias in Automatic Speech Recognition](https://mlsanity.com/paper/2103.15122/) (2021)
- [LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speech](https://mlsanity.com/paper/2104.11462/) (2021)
- [Learning Robust and Multilingual Speech Representations](https://mlsanity.com/paper/2001.11128/) (2020)
- [Gender Representation in French Broadcast Corpora and Its Impact on ASR Performance](https://mlsanity.com/paper/1908.08717/) (2019)
- [Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation](https://mlsanity.com/paper/2203.09866/) (2022)
- [Don't speak too fast: The impact of data bias on self-supervised speech models](https://mlsanity.com/paper/2110.07957/) (2021)
- [Evaluating Gender Bias in Speech Translation](https://mlsanity.com/paper/2010.14465/) (2020)
- [Att-HACK: An Expressive Speech Database with Social Attitudes](https://mlsanity.com/paper/2004.04410/) (2020)
- [Model-Based Approach for Measuring the Fairness in ASR](https://mlsanity.com/paper/2109.09061/) (2021)
- [Language technology practitioners as language managers: arbitrating data bias and predictive bias in ASR](https://mlsanity.com/paper/2202.12603/) (2022)

## Built on by (most cited first, 15 of 15)

- [Self-Supervised Multimodal Learning: A Survey](https://mlsanity.com/paper/2304.01008/) (2023)
- [Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models](https://mlsanity.com/paper/2407.06957/) (2024)
- [LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speech](https://mlsanity.com/paper/2309.05472/) (2023)
- [On the social bias of speech self-supervised models](https://mlsanity.com/paper/2406.04997/) (2024)
- [Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models](https://mlsanity.com/paper/2408.07665/) (2024)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [What Do Compressed Multilingual Machine Translation Models Forget?](https://mlsanity.com/paper/2205.10828/) (2022)
- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [Beyond the binary: Limitations and possibilities of gender-related speech technology research](https://mlsanity.com/paper/2409.13335/) (2024)
- [Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation](https://mlsanity.com/paper/2511.21517/) (2025)
- [Some voices are too common: Building fair speech recognition systems using the Common Voice dataset](https://mlsanity.com/paper/2306.03773/) (2023)
- [Efficient Compression of Multitask Multilingual Speech Models](https://mlsanity.com/paper/2405.00966/) (2024)
- [On the Role of Speech Data in Reducing Toxicity Detection Bias](https://mlsanity.com/paper/2411.08135/) (2024)
- [Bias and Fairness in Self-Supervised Acoustic Representations for Cognitive Impairment Detection](https://mlsanity.com/paper/2603.02937/) (2026)
- [Personalization Without Stereotyping: Do Speech LLMs Know When to Turn a Deaf Ear?](https://mlsanity.com/paper/2610.13550/) (2026)

## Abstract (arXiv)

Self-supervised models for speech processing emerged recently as popular foundation blocks in speech processing pipelines. These models are pre-trained on unlabeled audio data and then used in speech processing downstream tasks such as automatic speech recognition (ASR) or speech translation (ST). Since these models are now used in research and industrial systems alike, it becomes necessary to understand the impact caused by some features such as gender distribution within pre-training data. Using French as our investigation language, we train and compare gender-specific wav2vec 2.0 models against models containing different degrees of gender balance in their pre-training data. The comparison is performed by applying these models to two speech-to-text downstream tasks: ASR and ST. Results show the type of downstream integration matters. We observe lower overall performance using gender-specific pre-training before fine-tuning an end-to-end ASR system. However, when self-supervised models are used as feature extractors, the overall ASR and ST results follow more complex patterns in which the balanced pre-trained model does not necessarily lead to the best results. Lastly, our crude 'fairness' metric, the relative performance difference measured between female and male test sets, does not display a strong variation from balanced to gender-specific pre-trained wav2vec 2.0 models.