# Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder models

arXiv: https://arxiv.org/abs/1811.00403 · HTML: https://mlsanity.com/paper/1811.00403/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2018.

## Built on by (most cited first, 29 of 29)

- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Models](https://mlsanity.com/paper/2210.16043/) (2022)
- [Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation](https://mlsanity.com/paper/2103.10731/) (2021)
- [Generative Spoken Language Model based on continuous word-sized audio tokens](https://mlsanity.com/paper/2310.05224/) (2023)
- [SD-HuBERT: Sentence-Level Self-Distillation Induces Syllabic Organization in HuBERT](https://mlsanity.com/paper/2310.10803/) (2023)
- [DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexicon](https://mlsanity.com/paper/2206.11332/) (2022)
- [A Brief Overview of Unsupervised Neural Speech Representation Learning](https://mlsanity.com/paper/2203.01829/) (2022)
- [Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study](https://mlsanity.com/paper/2106.08686/) (2021)
- [How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddings](https://mlsanity.com/paper/2109.10179/) (2021)
- [Double Articulation Analyzer with Prosody for Unsupervised Word and Phoneme Discovery](https://mlsanity.com/paper/2103.08199/) (2021)
- [Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource language](https://mlsanity.com/paper/2106.12834/) (2021)
- [Interpreting intermediate convolutional layers of generative CNNs trained on waveforms](https://mlsanity.com/paper/2104.09489/) (2021)
- [A phonetic model of non-native spoken word processing](https://mlsanity.com/paper/2101.11332/) (2021)
- [Unsupervised Automatic Speech Recognition: A Review](https://mlsanity.com/paper/2106.04897/) (2021)
- [Towards hate speech detection in low-resource languages: Comparing ASR to acoustic word embeddings on Wolof and Swahili](https://mlsanity.com/paper/2306.00410/) (2023)
- [Visually grounded few-shot word learning in low-resource settings](https://mlsanity.com/paper/2306.11371/) (2023)
- [Analyzing the Representational Geometry of Acoustic Word Embeddings](https://mlsanity.com/paper/2301.03012/) (2023)
- [SCORE: Self-supervised Correspondence Fine-tuning for Improved Content Representations](https://mlsanity.com/paper/2403.06260/) (2024)
- [Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations](https://mlsanity.com/paper/2403.08738/) (2024)
- [Visually Grounded Speech Models have a Mutual Exclusivity Bias](https://mlsanity.com/paper/2403.13922/) (2024)
- [Integrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddings](https://mlsanity.com/paper/2209.06633/) (2022)
- [Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings](https://mlsanity.com/paper/2210.12857/) (2022)
- [Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Pooling](https://mlsanity.com/paper/2306.02153/) (2023)
- [Leveraging multilingual transfer for unsupervised semantic acoustic word embeddings](https://mlsanity.com/paper/2307.02083/) (2023)
- [LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Tasks](https://mlsanity.com/paper/2406.09153/) (2024)
- [Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network](https://mlsanity.com/paper/2207.14568/) (2022)
- [Visually grounded few-shot word acquisition with fewer shots](https://mlsanity.com/paper/2305.15937/) (2023)
- [XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words](https://mlsanity.com/paper/2310.05235/) (2023)
- [Joint Multimodal Contrastive Learning for Robust Spoken Term Detection and Keyword Spotting](https://mlsanity.com/paper/2512.14115/) (2025)

## Abstract (arXiv)

We investigate unsupervised models that can map a variable-duration speech segment to a fixed-dimensional representation. In settings where unlabelled speech is the only available resource, such acoustic word embeddings can form the basis for "zero-resource" speech search, discovery and indexing systems. Most existing unsupervised embedding methods still use some supervision, such as word or phoneme boundaries. Here we propose the encoder-decoder correspondence autoencoder (EncDec-CAE), which, instead of true word segments, uses automatically discovered segments: an unsupervised term discovery system finds pairs of words of the same unknown type, and the EncDec-CAE is trained to reconstruct one word given the other as input. We compare it to a standard encoder-decoder autoencoder (AE), a variational AE with a prior over its latent embedding, and downsampling. EncDec-CAE outperforms its closest competitor by 24% relative in average precision on two languages in a word discrimination task.