Leveraging multilingual transfer for unsupervised semantic acoustic word embeddings
Not on the map. Why?
Abstract (arXiv)
Acoustic word embeddings (AWEs) are fixed-dimensional vector representations of speech segments that encode phonetic content so that different realisations of the same word have similar embeddings. In this paper we explore semantic AWE modelling. These AWEs should not only capture phonetics but also the meaning of a word (similar to textual word embeddings). We consider the scenario where we only have untranscribed speech in a target language. We introduce a number of strategies leveraging a pre-trained multilingual AWE model -- a phonetic AWE model trained on labelled data from multiple languages excluding the target. Our best semantic AWE approach involves clustering word segments using the multilingual AWE model, deriving soft pseudo-word labels from the cluster centroids, and then training a Skipgram-like model on the soft vectors. In an intrinsic word similarity task measuring semantics, this multilingual transfer approach outperforms all previous semantic AWE methods. We also show -- for the first time -- that AWEs can be used for downstream semantic query-by-example search.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 5 Jul 2023.
Builds on
30 of 39 references.
References · 30
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
- 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
- 2017Word Translation Without Parallel Datacited by 480
- 2013Exploiting Similarities among Languages for Machine Translationcited by 187
- 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 170
- 2015Deep Multimodal Semantic Embeddings for Speech and Imagescited by 56
- 2018Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speechcited by 52
- 2017An embedded segmental K-means model for unsupervised segmentation and clustering of speechcited by 31
- 2018Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder modelscited by 29
18 more
- 2015Deep convolutional acoustic word embeddings using word-pair side informationcited by 27
- 2016Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approachescited by 25
- 2022Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoringcited by 24
- 2017Query-by-Example Search with Discriminative Neural Acoustic Word Embeddingscited by 22
- 2022Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Modelscited by 21
- 2017Semantic speech retrieval with a visually grounded model of untranscribed speechcited by 20
- 2021Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptationcited by 19
- 2021Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and wordscited by 18
- 2017Visually grounded learning of keyword prediction from untranscribed speechcited by 16
- 2020Multilingual Jointly Trained Acoustic and Written Word Embeddingscited by 16
- 2018Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrievalcited by 11
- 2020Improved acoustic word embeddings for zero-resource languages using multilingual transfercited by 11
- 2020A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddingscited by 10
- 2020Multilingual acoustic word embedding models for processing zero-resource languagescited by 9
- 2021Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource languagecited by 7
- 2022Unsupervised Word Segmentation using K Nearest Neighborscited by 5
- 2023Towards hate speech detection in low-resource languages: Comparing ASR to acoustic word embeddings on Wolof and Swahilicited by 5
- 2022Integrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddingscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.