# Predicting non-native speech perception using the Perceptual Assimilation Model and state-of-the-art acoustic models

arXiv: https://arxiv.org/abs/2205.15823 · HTML: https://mlsanity.com/paper/2205.15823/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 28 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 6 of 6)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [The Zero Resource Speech Challenge 2017](https://mlsanity.com/paper/1712.04313/) (2017)
- [Joint Masked CPC and CTC Training for ASR](https://mlsanity.com/paper/2011.00093/) (2020)
- [Perceptimatic: A human speech perception benchmark for unsupervised subword modelling](https://mlsanity.com/paper/2010.05961/) (2020)
- [The Perceptimatic English Benchmark for Speech Perception Models](https://mlsanity.com/paper/2005.03418/) (2020)

## Built on by (most cited first, 2 of 2)

- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge](https://mlsanity.com/paper/2210.15759/) (2022)

## Abstract (arXiv)

Our native language influences the way we perceive speech sounds, affecting our ability to discriminate non-native sounds. We compare two ideas about the influence of the native language on speech perception: the Perceptual Assimilation Model, which appeals to a mental classification of sounds into native phoneme categories, versus the idea that rich, fine-grained phonetic representations tuned to the statistics of the native language, are sufficient. We operationalize this idea using representations from two state-of-the-art speech models, a Dirichlet process Gaussian mixture model and the more recent wav2vec 2.0 model. We present a new, open dataset of French- and English-speaking participants' speech perception behaviour for 61 vowel sounds from six languages. We show that phoneme assimilation is a better predictor than fine-grained phonetic modelling, both for the discrimination behaviour as a whole, and for predicting differences in discriminability associated with differences in native language background. We also show that wav2vec 2.0, while not good at capturing the effects of native language on speech perception, is complementary to information about native phoneme assimilation, and provides a good model of low-level phonetic representations, supporting the idea that both categorical and fine-grained perception are used during speech perception.