mlsanityData to 14 Oct 2026

Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations

arXiv 2403.08738 ↗First posted 13 Mar 2024cs.CL cs.SD eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Acoustic word embeddings (AWEs) are vector representations of spoken words. An effective method for obtaining AWEs is the Correspondence Auto-Encoder (CAE). In the past, the CAE method has been associated with traditional MFCC features. Representations obtained from self-supervised learning (SSL)-based speech models such as HuBERT, Wav2vec2, etc., are outperforming MFCC in many downstream tasks. However, they have not been well studied in the context of learning AWEs. This work explores the effectiveness of CAE with SSL-based speech representations to obtain improved AWEs. Additionally, the capabilities of SSL-based speech models are explored in cross-lingual scenarios for obtaining AWEs. Experiments are conducted on five languages: Polish, Portuguese, Spanish, French, and English. HuBERT-based CAE model achieves the best results for word discrimination in all languages, despite Hu-BERT being pre-trained on English only. Also, the HuBERT-based CAE model works well in cross-lingual settings. It outperforms MFCC-based CAE models trained on the target languages when trained on one source language and tested on target languages.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

21 of 28 references.

References · 21

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  4. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  5. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  6. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  7. 2018Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder modelscited by 29
  8. 2017Query-by-Example Search with Discriminative Neural Acoustic Word Embeddingscited by 22
  9. 2022Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Modelscited by 21
  10. 2016Multi-view Recurrent Neural Acoustic Word Embeddingscited by 20
  11. 2022Learning Audio-Text Agreement for Open-vocabulary Keyword Spottingcited by 20
  12. 2020Improved acoustic word embeddings for zero-resource languages using multilingual transfercited by 11
9 more
  1. 2021Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Studycited by 8
  2. 2021How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddingscited by 8
  3. 2018Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Searchcited by 7
  4. 2020Analyzing autoencoder-based acoustic word embeddingscited by 7
  5. 2020Acoustic span embeddings for multilingual query-by-example searchcited by 7
  6. 2023Towards hate speech detection in low-resource languages: Comparing ASR to acoustic word embeddings on Wolof and Swahilicited by 5
  7. 2024SCORE: Self-supervised Correspondence Fine-tuning for Improved Content Representationscited by 3
  8. 2022Integrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddingscited by 2
  9. 2023Self-Supervised Acoustic Word Embedding Learning via Correspondence Transformer Encodercited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 2.

3 papers.

  1. 2024LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Taskscited by 2
  2. 2026Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluationcited by 1
  3. 2026Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representationscited by 0