mlsanityData to 7 Oct 2026

Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource language

arXiv 2106.12834 ↗First posted 24 Jun 2021cs.CL cs.SD eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

Acoustic word embedding models map variable duration speech segments to fixed dimensional vectors, enabling efficient speech search and discovery. Previous work explored how embeddings can be obtained in zero-resource settings where no labelled data is available in the target language. The current best approach uses transfer learning: a single supervised multilingual model is trained using labelled data from multiple well-resourced languages and then applied to a target zero-resource language (without fine-tuning). However, it is still unclear how the specific choice of training languages affect downstream performance. Concretely, here we ask whether it is beneficial to use training languages related to the target. Using data from eleven languages spoken in Southern Africa, we experiment with adding data from different language families while controlling for the amount of data per language. In word discrimination and query-by-example search evaluations, we show that training on languages from the same family gives large improvements. Through finer-grained analysis, we show that training on even just a single related language gives the largest gain. We also find that adding data from unrelated languages generally doesn't hurt performance.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 24 Jun 2021.

Builds on

31 of 44 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speechcited by 52
  4. 2020Masakhane -- Machine Translation For Africacited by 48
  5. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
  6. 2017An embedded segmental K-means model for unsupervised segmentation and clustering of speechcited by 31
  7. 2018Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder modelscited by 29
  8. 2015Deep convolutional acoustic word embeddings using word-pair side informationcited by 27
  9. 2017Query-by-Example Search with Discriminative Neural Acoustic Word Embeddingscited by 22
  10. 2020Evaluating the reliability of acoustic speech embeddingscited by 21
  11. 2016Multi-view Recurrent Neural Acoustic Word Embeddingscited by 20
  12. 2017End-to-End ASR-free Keyword Search from Speechcited by 19
19 more
  1. 2021Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptationcited by 19
  2. 2018Segmental Audio Word2Vec: Representing Utterances as Sequences of Vectors with Applications in Spoken Term Detectioncited by 17
  3. 2020Multilingual Jointly Trained Acoustic and Written Word Embeddingscited by 16
  4. 2020Unsupervised Discovery of Recurring Speech Patterns Using Probabilistic Adaptive Metricscited by 16
  5. 2019Audio-Linguistic Embeddings for Spoken Sentencescited by 15
  6. 2019Acoustically Grounded Word Embeddings for Improved Acoustics-to-Word Speech Recognitioncited by 15
  7. 2018Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrievalcited by 11
  8. 2019Semantic query-by-example speech search using visual groundingcited by 11
  9. 2020Improved acoustic word embeddings for zero-resource languages using multilingual transfercited by 11
  10. 2019Bayesian Subspace Hidden Markov Model for Acoustic Unit Discoverycited by 9
  11. 2020Multilingual acoustic word embedding models for processing zero-resource languagescited by 9
  12. 2018Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Searchcited by 7
  13. 2020Acoustic span embeddings for multilingual query-by-example searchcited by 7
  14. 2018Multilingual and Unsupervised Subword Modeling for Zero-Resource Languagescited by 6
  15. 2019Additional Shared Decoder on Siamese Multi-view Encoders for Learning Acoustic Word Embeddingscited by 6
  16. 2019Contextual Joint Factor Acoustic Embeddingscited by 3
  17. 2018Improved Audio Embeddings by Adjacency-Based Clustering with Applications in Spoken Term Detectioncited by 2
  18. 2019SylNet: An Adaptable End-to-End Syllable Count Estimator for Speechcited by 2
  19. 2020Acoustic Word Embedding System for Code-Switching Query-by-example Spoken Term Detectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 3; 2024: 0; 2025: 0; 2026: 1.

7 papers.

  1. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  2. 2021How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddingscited by 8
  3. 2023Towards hate speech detection in low-resource languages: Comparing ASR to acoustic word embeddings on Wolof and Swahilicited by 5
  4. 2022Integrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddingscited by 2
  5. 2023Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Poolingcited by 2
  6. 2023Leveraging multilingual transfer for unsupervised semantic acoustic word embeddingscited by 2
  7. 2026Low resource cross-modal alignment using HGNN to enhance speech representationcited by 0