# Do self-supervised speech models develop human-like perception biases?

arXiv: https://arxiv.org/abs/2205.15819 · HTML: https://mlsanity.com/paper/2205.15819/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 14 of 14)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Generative Spoken Language Modeling from Raw Audio](https://mlsanity.com/paper/2102.01192/) (2021)
- [Unsupervised pretraining transfers well across languages](https://mlsanity.com/paper/2002.02848/) (2020)
- [Self-training and Pre-training are Complementary for Speech Recognition](https://mlsanity.com/paper/2010.11430/) (2020)
- [The Zero Resource Speech Challenge 2021: Spoken language modelling](https://mlsanity.com/paper/2104.14700/) (2021)
- [Evaluating computational models of infant phonetic learning across languages](https://mlsanity.com/paper/2008.02888/) (2020)
- [Perceptimatic: A human speech perception benchmark for unsupervised subword modelling](https://mlsanity.com/paper/2010.05961/) (2020)
- [The Perceptimatic English Benchmark for Speech Perception Models](https://mlsanity.com/paper/2005.03418/) (2020)

## Built on by (most cited first, 28 of 28)

- [Decoding speech perception from non-invasive brain recordings](https://mlsanity.com/paper/2208.12266/) (2022)
- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [How Linguistics Learned to Stop Worrying and Love the Language Models](https://mlsanity.com/paper/2501.17047/) (2025)
- [Probing self-supervised speech models for phonetic and phonemic information: a case study in aspiration](https://mlsanity.com/paper/2306.06232/) (2023)
- [Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge](https://mlsanity.com/paper/2210.15759/) (2022)
- [Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0](https://mlsanity.com/paper/2407.03005/) (2024)
- [On the social bias of speech self-supervised models](https://mlsanity.com/paper/2406.04997/) (2024)
- [Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers](https://mlsanity.com/paper/2310.09925/) (2023)
- [fastabx: A library for efficient computation of ABX discriminability](https://mlsanity.com/paper/2505.02692/) (2025)
- [What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training](https://mlsanity.com/paper/2506.00981/) (2025)
- [Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models](https://mlsanity.com/paper/2506.10855/) (2025)
- [The Lipreading Gap: Do VSR Models Perceive Visual Speech Like Human Lipreaders?](https://mlsanity.com/paper/2606.07435/) (2026)
- [A predictive learning model can simulate temporal dynamics and context effects found in neural representations of continuous speech](https://mlsanity.com/paper/2405.08237/) (2024)
- [LID Models are Actually Accent Classifiers: Implications and Solutions for LID on Accented Speech](https://mlsanity.com/paper/2506.00628/) (2025)
- [Tracking the emergence of linguistic structure in self-supervised models learning from speech](https://mlsanity.com/paper/2604.02043/) (2026)
- [Do Self-Supervised Speech Models Exhibit the Critical Period Effects in Language Acquisition?](https://mlsanity.com/paper/2508.21210/) (2025)
- [Language models as tools for investigating the distinction between possible and impossible natural languages](https://mlsanity.com/paper/2512.09394/) (2025)
- [Large language models have learned to use language](https://mlsanity.com/paper/2512.12447/) (2025)
- [Large language models are not about natural language](https://mlsanity.com/paper/2512.13441/) (2025)
- [Linguists should learn to love speech-based deep learning models](https://mlsanity.com/paper/2512.14506/) (2025)
- [Are Language Models Models?](https://mlsanity.com/paper/2601.10421/) (2026)
- [Acoustic and perceptual differences between standard and accented speech and their voice clones](https://mlsanity.com/paper/2604.01562/) (2026)
- [Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated Probabilities](https://mlsanity.com/paper/2604.09466/) (2026)
- [You Can't Fight in Here! This is BBS!](https://mlsanity.com/paper/2604.09501/) (2026)
- [Not-So-Strange Love: Language Models and Generative Linguistic Theories are More Compatible than They Appear](https://mlsanity.com/paper/2605.10061/) (2026)
- [Pretrained self-supervised speech models can recognize unseen consonants](https://mlsanity.com/paper/2606.11542/) (2026)
- [Perceptual compensation for tonal context in self-supervised speech models](https://mlsanity.com/paper/2606.17835/) (2026)
- [The Learning Objective Governs Perceptual Narrowing: A Cross-Lingual, Layer-Wise, Ten-Seed Study of Self-Supervised Speech Encoders](https://mlsanity.com/paper/2608.00507/) (2026)

## Abstract (arXiv)

Self-supervised models for speech processing form representational spaces without using any external labels. Increasingly, they appear to be a feasible way of at least partially eliminating costly manual annotations, a problem of particular concern for low-resource languages. But what kind of representational spaces do these models construct? Human perception specializes to the sounds of listeners' native languages. Does the same thing happen in self-supervised models? We examine the representational spaces of three kinds of state-of-the-art self-supervised models: wav2vec 2.0, HuBERT and contrastive predictive coding (CPC), and compare them with the perceptual spaces of French-speaking and English-speaking human listeners, both globally and taking account of the behavioural differences between the two language groups. We show that the CPC model shows a small native language effect, but that wav2vec 2.0 and HuBERT seem to develop a universal speech perception space which is not language specific. A comparison against the predictions of supervised phone recognisers suggests that all three self-supervised models capture relatively fine-grained perceptual phenomena, while supervised models are better at capturing coarser, phone-level, effects of listeners' native language, on perception.