mlsanityData to 14 Oct 2026

Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation

arXiv 2103.10731 ↗First posted 19 Mar 2021cs.CL eess.AScited by 19

Not on the map. Why?

Abstract (arXiv)

Acoustic word embeddings (AWEs) are fixed-dimensional representations of variable-length speech segments. For zero-resource languages where labelled data is not available, one AWE approach is to use unsupervised autoencoder-based recurrent models. Another recent approach is to use multilingual transfer: a supervised AWE model is trained on several well-resourced languages and then applied to an unseen zero-resource language. We consider how a recent contrastive learning loss can be used in both the purely unsupervised and multilingual transfer settings. Firstly, we show that terms from an unsupervised term discovery system can be used for contrastive self-supervision, resulting in improvements over previous unsupervised monolingual AWE models. Secondly, we consider how multilingual AWE models can be adapted to a specific zero-resource language using discovered terms. We find that self-supervised contrastive adaptation outperforms adapted multilingual correspondence autoencoder and Siamese AWE models, giving the best overall results in a word discrimination task on six zero-resource languages.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 19 Mar 2021.

Builds on

42 of 61 references.

References · 42

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  4. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  5. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  6. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 786
  7. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 340
  8. 2017Multi-task Self-Supervised Visual Learningcited by 200
  9. 2020Multi-task self-supervised learning for Robust Speech Recognitioncited by 130
  10. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  11. 2019Learning Problem-agnostic Speech Representations from Multiple Self-supervised Taskscited by 87
  12. 2019A critical analysis of self-supervision, or what we can learn from a single imagecited by 76
30 more
  1. 2018Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speechcited by 52
  2. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
  3. 2019The Zero Resource Speech Challenge 2019: TTS without Tcited by 38
  4. 2020Unsupervised Pre-training of Bidirectional Speech Encoders via Masked Reconstructioncited by 38
  5. 2017An embedded segmental K-means model for unsupervised segmentation and clustering of speechcited by 31
  6. 2018Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder modelscited by 29
  7. 2015Deep convolutional acoustic word embeddings using word-pair side informationcited by 27
  8. 2016Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approachescited by 25
  9. 2020Evaluating the reliability of acoustic speech embeddingscited by 21
  10. 2016Multi-view Recurrent Neural Acoustic Word Embeddingscited by 20
  11. 2017End-to-End ASR-free Keyword Search from Speechcited by 19
  12. 2018Segmental Audio Word2Vec: Representing Utterances as Sequences of Vectors with Applications in Spoken Term Detectioncited by 17
  13. 2020Multilingual Jointly Trained Acoustic and Written Word Embeddingscited by 16
  14. 2020Unsupervised Discovery of Recurring Speech Patterns Using Probabilistic Adaptive Metricscited by 16
  15. 2019Audio-Linguistic Embeddings for Spoken Sentencescited by 15
  16. 2019Acoustically Grounded Word Embeddings for Improved Acoustics-to-Word Speech Recognitioncited by 15
  17. 2018Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrievalcited by 11
  18. 2020Improved acoustic word embeddings for zero-resource languages using multilingual transfercited by 11
  19. 2019Bayesian Subspace Hidden Markov Model for Acoustic Unit Discoverycited by 9
  20. 2020Multilingual acoustic word embedding models for processing zero-resource languagescited by 9
  21. 2018Learning Acoustic Word Embeddings with Temporal Context for Query-by-Example Speech Searchcited by 7
  22. 2020Analyzing autoencoder-based acoustic word embeddingscited by 7
  23. 2018Multilingual and Unsupervised Subword Modeling for Zero-Resource Languagescited by 6
  24. 2019Additional Shared Decoder on Siamese Multi-view Encoders for Learning Acoustic Word Embeddingscited by 6
  25. 2016An Unsupervised Probability Model for Speech-to-Translation Alignment of Low-Resource Languagescited by 3
  26. 2019Contextual Joint Factor Acoustic Embeddingscited by 3
  27. 2018Improved Audio Embeddings by Adjacency-Based Clustering with Applications in Spoken Term Detectioncited by 2
  28. 2019SylNet: An Adaptable End-to-End Syllable Count Estimator for Speechcited by 2
  29. 2020Acoustic Word Embedding System for Code-Switching Query-by-example Spoken Term Detectioncited by 2
  30. 2020Automatic Detection of Phonological Errors in Child Speech Using Siamese Recurrent Autoencodercited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 8; 2024: 0; 2025: 2; 2026: 1.

19 papers.

Sort:

By area:

  1. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  2. 2023Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognitioncited by 26
  3. 2022LAE: Language-Aware Encoder for Monolingual and Multilingual ASRcited by 22
  4. 2022Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Modelscited by 21
  5. 2023Generative Spoken Language Model based on continuous word-sized audio tokenscited by 19
  6. 2022DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexiconcited by 12
  7. 2022A Brief Overview of Unsupervised Neural Speech Representation Learningcited by 9
  8. 2021Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource languagecited by 7
  9. 2022Speech Sequence Embeddings using Nearest Neighbors Contrastive Learningcited by 7
  10. 2022Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognitioncited by 5
  11. 2023Towards hate speech detection in low-resource languages: Comparing ASR to acoustic word embeddings on Wolof and Swahilicited by 5
  12. 2023Visually grounded few-shot word learning in low-resource settingscited by 4
7 more
  1. 2023Analyzing the Representational Geometry of Acoustic Word Embeddingscited by 3
  2. 2023Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Poolingcited by 2
  3. 2023Leveraging multilingual transfer for unsupervised semantic acoustic word embeddingscited by 2
  4. 2023Supervised Acoustic Embeddings And Their Transferability Across Languagescited by 1
  5. 2025Towards few-shot isolated word reading assessmentcited by 1
  6. 2025Joint Multimodal Contrastive Learning for Robust Spoken Term Detection and Keyword Spottingcited by 0
  7. 2026Low resource cross-modal alignment using HGNN to enhance speech representationcited by 0