mlsanityData to 14 Oct 2026

Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Models

arXiv 2210.16043 ↗First posted 28 Oct 2022cs.CL cs.SD eess.AScited by 21

Not on the map. Why?

Abstract (arXiv)

Given the strong results of self-supervised models on various tasks, there have been surprisingly few studies exploring self-supervised representations for acoustic word embeddings (AWE), fixed-dimensional vectors representing variable-length spoken word segments. In this work, we study several pre-trained models and pooling methods for constructing AWEs with self-supervised representations. Owing to the contextualized nature of self-supervised representations, we hypothesize that simple pooling methods, such as averaging, might already be useful for constructing AWEs. When evaluating on a standard word discrimination task, we find that HuBERT representations with mean-pooling rival the state of the art on English AWEs. More surprisingly, despite being trained only on English, HuBERT representations evaluated on Xitsonga, Mandarin, and French consistently outperform the multilingual model XLSR-53 (as well as Wav2Vec 2.0 trained on English).

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 28 Oct 2022.

Builds on

20 of 32 references.

References · 20

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  5. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  6. 2021Unsupervised Speech Recognitioncited by 148
  7. 2017The Zero Resource Speech Challenge 2017cited by 45
  8. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 45
  9. 2017An embedded segmental K-means model for unsupervised segmentation and clustering of speechcited by 31
  10. 2018Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder modelscited by 29
  11. 2015Deep convolutional acoustic word embeddings using word-pair side informationcited by 27
  12. 2016A segmental framework for fully-unsupervised large-vocabulary speech recognitioncited by 27
8 more
  1. 2016Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approachescited by 25
  2. 2017Query-by-Example Search with Discriminative Neural Acoustic Word Embeddingscited by 22
  3. 2021Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptationcited by 19
  4. 2020Multilingual Jointly Trained Acoustic and Written Word Embeddingscited by 16
  5. 2020A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddingscited by 12
  6. 2020A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddingscited by 10
  7. 2022Speech Sequence Embeddings using Nearest Neighbors Contrastive Learningcited by 7
  8. 2017Unsupervised neural and Bayesian models for zero-resource speech processingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 7; 2025: 8; 2026: 1.

21 papers.

Sort:

  1. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  2. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  3. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  4. 2023Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspacescited by 20
  5. 2025Emergent morpho-phonological representations in self-supervised speech modelscited by 11
  6. 2024Crossmodal ASR Error Correction with Discrete Speech Unitscited by 9
  7. 2024Layer-Wise Analysis of Self-Supervised Acoustic Word Embeddings: A Study on Speech Emotion Recognitioncited by 8
  8. 2024What Do Speech Foundation Models Not Learn About Speech?cited by 8
  9. 2024Unsupervised Word Discovery: Boundary Detection with Clustering vs. Dynamic Programmingcited by 7
  10. 2025Segmentation-Variant Codebooks for Preservation of Paralinguistic and Prosodic Informationcited by 4
  11. 2025Should Top-Down Clustering Affect Boundaries in Unsupervised Word Discovery?cited by 4
  12. 2025Iterative refinement, not training objective, makes HuBERT behave differently from wav2vec 2.0cited by 4
9 more
  1. 2025Unsupervised lexicon learning from speech is limited by representations rather than clusteringcited by 4
  2. 2024Revisiting speech segmentation and lexicon learning with better featurescited by 3
  3. 2024Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representationscited by 3
  4. 2025Discrete Speech Unit Extraction via Independent Component Analysiscited by 3
  5. 2023Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Poolingcited by 2
  6. 2023Leveraging multilingual transfer for unsupervised semantic acoustic word embeddingscited by 2
  7. 2025Enhancing Few-shot Keyword Spotting Performance through Pre-Trained Self-supervised Speech Modelscited by 1
  8. 2025Do Self-Supervised Speech Models Exhibit the Critical Period Effects in Language Acquisition?cited by 0
  9. 2026Learning task-specific subspaces via interventional post-training of speech foundation modelscited by 0