mlsanityData to 14 Oct 2026

Do speech foundation models really learn words?

arXiv 2609.10434 ↗First posted 9 Sep 2026cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Self-supervised speech foundation models are now used in a wide array of downstream applications, including traditional speech recognition and as the basis for tokens in speech-aware language models. Attempts to understand their usefulness have largely focused on probing their representations' ability to discriminate phonemes and words. However, discriminative ability for words need not imply specialized representation of words per se. Good discrimination of words may be explained by good encoding of word form (phonemes) rather than form-independent word representations encoding identity or syntactic/semantic properties. By partialling out phoneme information using residualization, we show that, in later layers, HuBERT and wav2vec 2.0 do in general learn representations which encode words with reasonable fidelity independently of local phonetic content. We show that this simple approach to disentanglement can enhance higher-order linguistic information in word discovery tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 9 Sep 2026.

Builds on

22 of 29 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  6. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  7. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  8. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  9. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  10. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  11. 2022Joint processing of linguistic properties in brains and language modelscited by 37
  12. 2023Probing self-supervised speech models for phonetic and phonemic information: a case study in aspirationcited by 36
10 more
  1. 2022SpeechSplit 2.0: Unsupervised speech disentanglement for voice conversion Without tuning autoencoder Bottleneckscited by 28
  2. 2021Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processingcited by 26
  3. 2023Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspacescited by 20
  4. 2023Speech language models lack important brain-relevant semanticscited by 19
  5. 2023Wave to Syntax: Probing spoken language models for syntaxcited by 18
  6. 2025SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervisioncited by 8
  7. 2024Unsupervised Word Discovery: Boundary Detection with Clustering vs. Dynamic Programmingcited by 7
  8. 2025Should Top-Down Clustering Affect Boundaries in Unsupervised Word Discovery?cited by 4
  9. 2025Discrete Speech Unit Extraction via Independent Component Analysiscited by 3
  10. 2024Measuring Orthogonality in Representations of Generative Modelscited by 1

Built on by

Not cited yet.