mlsanityData to 7 Oct 2026

Probing phoneme, language and speaker information in unsupervised speech representations

arXiv 2203.16193 ↗First posted 30 Mar 2022eess.AS cs.SDcited by 31

Not on the map. Why?

Abstract (arXiv)

Unsupervised models of representations based on Contrastive Predictive Coding (CPC)[1] are primarily used in spoken language modelling in that they encode phonetic information. In this study, we ask what other types of information are present in CPC speech representations. We focus on three categories: phone class, gender and language, and compare monolingual and bilingual models. Using qualitative and quantitative tools, we find that both gender and phone class information are present in both types of models. Language information, however, is very salient in the bilingual model only, suggesting CPC models learn to discriminate languages when trained on multiple languages. Some language information can also be retrieved from monolingual models, but it is more diffused across all features. These patterns hold when analyses are carried on the discrete units from a downstream clustering model. However, although there is no effect of the number of target clusters on phone class and language information, more gender information is encoded with more clusters. Finally, we find that there is some cost to being exposed to two languages on a downstream phoneme discrimination task.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Mar 2022.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 8; 2024: 5; 2025: 11; 2026: 5.

31 papers.

Sort:

  1. 2023Analysing Discrete Self Supervised Speech Representation for Spoken Language Modelingcited by 47
  2. 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challengecited by 34
  3. 2024Encoding of lexical tone in self-supervised models of spoken languagecited by 22
  4. 2023EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Modelscited by 21
  5. 2023Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspacescited by 19
  6. 2022Opening the Black Box of wav2vec Feature Encodercited by 18
  7. 2023Wave to Syntax: Probing spoken language models for syntaxcited by 18
  8. 2024Cross-Lingual Transfer Learning for Speech Translationcited by 11
  9. 2024How do Multimodal Foundation Models Encode Text and Speech? An Analysis of Cross-Lingual and Cross-Modal Representationscited by 8
  10. 2025The Role of Prosody in Spoken Question Answeringcited by 6
  11. 2025SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervisioncited by 6
  12. 2025Effective Context in Neural Speech Modelscited by 5
19 more
  1. 2025Layer-wise Minimal Pair Probing Reveals Contextual Grammatical-Conceptual Hierarchy in Speech Representationscited by 5
  2. 2025Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translationcited by 5
  3. 2025Different Speech Translation Models Encode and Translate Speaker Gender Differentlycited by 3
  4. 2025Analyzing the relationships between pretraining language, phonetic, tonal, and speaker information in self-supervised speech modelscited by 3
  5. 2023Phone and speaker spatial organization in self-supervised speech representationscited by 2
  6. 2023How Generative Spoken Language Modeling Encodes Noisy Speech: Investigation from Phonetics to Syntacticscited by 2
  7. 2023Big model only for hard audios: Sample dependent Whisper model selection for efficient inferencescited by 2
  8. 2024A predictive learning model can simulate temporal dynamics and context effects found in neural representations of continuous speechcited by 2
  9. 2024Phonetic Enhanced Language Modeling for Text-to-Speech Synthesiscited by 2
  10. 2025Echoes of Phonetics: Unveiling Relevant Acoustic Cues for ASR via Feature Attributioncited by 2
  11. 2026Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networkscited by 2
  12. 2023From `Snippet-lects' to Doculects and Dialects: Leveraging Neural Representations of Speech for Placing Audio Signals in a Language Landscapecited by 1
  13. 2025MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discoverycited by 1
  14. 2026Represented but Ignored: A Causal Account of Prosodic Underuse in Audio-Language Modelscited by 1
  15. 2025Probing for Phonology in Self-Supervised Speech Representations: A Case Study on Accent Perceptioncited by 0
  16. 2025Sparse deepfake detection promotes better disentanglementcited by 0
  17. 2026DiscoPhon: Benchmarking the Unsupervised Discovery of Phoneme Inventories With Discrete Speech Unitscited by 0
  18. 2026Probing in the Wild: A Case Study of Self-Supervised Speech Representations on Mandarin Sub-dialects with Unsupervised Articulatory Analysiscited by 0
  19. 2026Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Modelscited by 0