mlsanityData to 14 Oct 2026

Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0

arXiv 2508.08110 ↗First posted 11 Aug 2025cs.CL cs.SD eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Self-supervised models for speech representation learning now see widespread use for their versatility and performance on downstream tasks, but the effect of model architecture on the linguistic information learned in their representations remains under-studied. This study investigates two such models, HuBERT and wav2vec 2.0, and minimally compares two of their architectural differences: training objective and iterative pseudo-label refinement through multiple training iterations. We find that differences in canonical correlation of hidden representations to word identity, phoneme identity, and speaker identity are explained by training iteration, not training objective. We suggest that future work investigate the reason for the effectiveness of iterative refinement in encoding linguistic information in self-supervised speech representations.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

23 of 26 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  6. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  7. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  8. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
  9. 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
  10. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  11. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  12. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
11 more
  1. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  2. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  3. 2023Probing self-supervised speech models for phonetic and phonemic information: a case study in aspirationcited by 36
  4. 2020Similarity Analysis of Self-Supervised Speech Representationscited by 26
  5. 2021Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processingcited by 26
  6. 2022Are discrete units necessary for Spoken Language Modeling?cited by 24
  7. 2022Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Modelscited by 21
  8. 2023What Can an Accent Identifier Learn? Probing Phonetic and Prosodic Information in a Wav2vec2-based Accent Identification Modelcited by 18
  9. 2024Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representationscited by 14
  10. 2023Pushing the Limits of Unsupervised Unit Discovery for SSL Speech Representationcited by 7
  11. 2024Probing Self-supervised Learning Models with Target Speech Extractioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Representation Potentials of Foundation Models for Multimodal Alignment: A Surveycited by 11
  2. 2026Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspacescited by 8
  3. 2026Mind the Shift: Using Delta SSL Embeddings to Enhance Child ASRcited by 3
  4. 2025Better audio representations are more brain-like: linking model-brain alignment with performance in downstream auditory taskscited by 0