# Probing phoneme, language and speaker information in unsupervised speech representations

arXiv: https://arxiv.org/abs/2203.16193 · HTML: https://mlsanity.com/paper/2203.16193/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2022.

## Built on by (most cited first, 30 of 33)

- [Analysing Discrete Self Supervised Speech Representation for Spoken Language Modeling](https://mlsanity.com/paper/2301.00591/) (2023)
- [Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge](https://mlsanity.com/paper/2210.15759/) (2022)
- [Encoding of lexical tone in self-supervised models of spoken language](https://mlsanity.com/paper/2403.16865/) (2024)
- [EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Models](https://mlsanity.com/paper/2312.14069/) (2023)
- [Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspaces](https://mlsanity.com/paper/2305.12464/) (2023)
- [Opening the Black Box of wav2vec Feature Encoder](https://mlsanity.com/paper/2210.15386/) (2022)
- [Wave to Syntax: Probing spoken language models for syntax](https://mlsanity.com/paper/2305.18957/) (2023)
- [Cross-Lingual Transfer Learning for Speech Translation](https://mlsanity.com/paper/2407.01130/) (2024)
- [How do Multimodal Foundation Models Encode Text and Speech? An Analysis of Cross-Lingual and Cross-Modal Representations](https://mlsanity.com/paper/2411.17666/) (2024)
- [SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervision](https://mlsanity.com/paper/2512.20308/) (2025)
- [The Role of Prosody in Spoken Question Answering](https://mlsanity.com/paper/2502.05389/) (2025)
- [Effective Context in Neural Speech Models](https://mlsanity.com/paper/2505.22487/) (2025)
- [Layer-wise Minimal Pair Probing Reveals Contextual Grammatical-Conceptual Hierarchy in Speech Representations](https://mlsanity.com/paper/2509.15655/) (2025)
- [Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation](https://mlsanity.com/paper/2511.21517/) (2025)
- [Different Speech Translation Models Encode and Translate Speaker Gender Differently](https://mlsanity.com/paper/2506.02172/) (2025)
- [Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech models](https://mlsanity.com/paper/2506.10855/) (2025)
- [Phone and speaker spatial organization in self-supervised speech representations](https://mlsanity.com/paper/2302.14055/) (2023)
- [How Generative Spoken Language Modeling Encodes Noisy Speech: Investigation from Phonetics to Syntactics](https://mlsanity.com/paper/2306.00697/) (2023)
- [Big model only for hard audios: Sample dependent Whisper model selection for efficient inferences](https://mlsanity.com/paper/2309.12712/) (2023)
- [A predictive learning model can simulate temporal dynamics and context effects found in neural representations of continuous speech](https://mlsanity.com/paper/2405.08237/) (2024)
- [Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis](https://mlsanity.com/paper/2406.02009/) (2024)
- [Echoes of Phonetics: Unveiling Relevant Acoustic Cues for ASR via Feature Attribution](https://mlsanity.com/paper/2506.02181/) (2025)
- [Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks](https://mlsanity.com/paper/2601.18617/) (2026)
- [From `Snippet-lects' to Doculects and Dialects: Leveraging Neural Representations of Speech for Placing Audio Signals in a Language Landscape](https://mlsanity.com/paper/2305.18602/) (2023)
- [MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery](https://mlsanity.com/paper/2512.19612/) (2025)
- [Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Models](https://mlsanity.com/paper/2608.06409/) (2026)
- [Represented but Ignored: A Causal Account of Prosodic Underuse in Audio-Language Models](https://mlsanity.com/paper/2608.19211/) (2026)
- [Probing for Phonology in Self-Supervised Speech Representations: A Case Study on Accent Perception](https://mlsanity.com/paper/2506.17542/) (2025)
- [Sparse deepfake detection promotes better disentanglement](https://mlsanity.com/paper/2510.05696/) (2025)
- [DiscoPhon: Benchmarking the Unsupervised Discovery of Phoneme Inventories With Discrete Speech Units](https://mlsanity.com/paper/2603.18612/) (2026)

## Abstract (arXiv)

Unsupervised models of representations based on Contrastive Predictive Coding (CPC)[1] are primarily used in spoken language modelling in that they encode phonetic information. In this study, we ask what other types of information are present in CPC speech representations. We focus on three categories: phone class, gender and language, and compare monolingual and bilingual models. Using qualitative and quantitative tools, we find that both gender and phone class information are present in both types of models. Language information, however, is very salient in the bilingual model only, suggesting CPC models learn to discriminate languages when trained on multiple languages. Some language information can also be retrieved from monolingual models, but it is more diffused across all features. These patterns hold when analyses are carried on the discrete units from a downstream clustering model. However, although there is no effect of the number of target clusters on phone class and language information, more gender information is encoded with more clusters. Finally, we find that there is some cost to being exposed to two languages on a downstream phoneme discrimination task.