# Self-supervised Speech Representations Still Struggle with African American Vernacular English

arXiv: https://arxiv.org/abs/2408.14262 · HTML: https://mlsanity.com/paper/2408.14262/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 14 of 14)

- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [Probing self-supervised speech models for phonetic and phonemic information: a case study in aspiration](https://mlsanity.com/paper/2306.06232/) (2023)
- [Don't speak too fast: The impact of data bias on self-supervised speech models](https://mlsanity.com/paper/2110.07957/) (2021)
- [Opening the Black Box of wav2vec Feature Encoder](https://mlsanity.com/paper/2210.15386/) (2022)
- [Accented Speech Recognition: Benchmarking, Pre-training, and Diverse Data](https://mlsanity.com/paper/2205.08014/) (2022)
- [Neural Representations for Modeling Variation in Speech](https://mlsanity.com/paper/2011.12649/) (2020)
- [Quantifying Language Variation Acoustically with Few Resources](https://mlsanity.com/paper/2205.02694/) (2022)
- [Can Voice Assistants Be Microaggressors? Cross-Race Psychological Responses to Failures of Automatic Speech Recognition](https://mlsanity.com/paper/2302.12326/) (2023)
- [Automatic Dialect Density Estimation for African American English](https://mlsanity.com/paper/2204.00967/) (2022)
- [Improving Speech Recognition for African American English With Audio Classification](https://mlsanity.com/paper/2309.09996/) (2023)

## Built on by (most cited first, 6 of 6)

- [Voxlect: A Speech Foundation Model Benchmark for Modeling Dialects and Regional Languages Around the Globe](https://mlsanity.com/paper/2508.01691/) (2025)
- [The ML-SUPERB 2.0 Challenge: Towards Inclusive ASR Benchmarking for All Language Varieties](https://mlsanity.com/paper/2509.07139/) (2025)
- [Swivuriso: The South African Next Voices Multilingual Speech Dataset](https://mlsanity.com/paper/2512.02201/) (2025)
- [Automatic Speech Recognition of African American English: Lexical and Contextual Effects](https://mlsanity.com/paper/2506.06888/) (2025)
- [Dialetto, ma Quanto Dialetto? Transcribing and Evaluating Dialects on a Continuum](https://mlsanity.com/paper/2410.14589/) (2024)
- [A Sociophonetic Analysis of Racial Bias in Commercial ASR Systems Using the Pacific Northwest English Corpus](https://mlsanity.com/paper/2510.22495/) (2025)

## Abstract (arXiv)

Underperformance of ASR systems for speakers of African American Vernacular English (AAVE) and other marginalized language varieties is a well-documented phenomenon, and one that reinforces the stigmatization of these varieties. We investigate whether or not the recent wave of Self-Supervised Learning (SSL) speech models can close the gap in ASR performance between AAVE and Mainstream American English (MAE). We evaluate four SSL models (wav2vec 2.0, HuBERT, WavLM, and XLS-R) on zero-shot Automatic Speech Recognition (ASR) for these two varieties and find that these models perpetuate the bias in performance against AAVE. Additionally, the models have higher word error rates on utterances with more phonological and morphosyntactic features of AAVE. Despite the success of SSL speech models in improving ASR for low resource varieties, SSL pre-training alone may not bridge the gap between AAVE and MAE. Our code is publicly available at https://github.com/cmu-llab/s3m-aave.