# A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddings

arXiv: https://arxiv.org/abs/2012.02221 · HTML: https://mlsanity.com/paper/2012.02221/

## Key facts

- Cited by 12 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 12 of 12)

- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Models](https://mlsanity.com/paper/2210.16043/) (2022)
- [Generative Spoken Language Model based on continuous word-sized audio tokens](https://mlsanity.com/paper/2310.05224/) (2023)
- [Layer-Wise Analysis of Self-Supervised Acoustic Word Embeddings: A Study on Speech Emotion Recognition](https://mlsanity.com/paper/2402.02617/) (2024)
- [Configurable Privacy-Preserving Automatic Speech Recognition](https://mlsanity.com/paper/2104.00766/) (2021)
- [Unsupervised Automatic Speech Recognition: A Review](https://mlsanity.com/paper/2106.04897/) (2021)
- [Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Pooling](https://mlsanity.com/paper/2306.02153/) (2023)
- [Supervised Acoustic Embeddings And Their Transferability Across Languages](https://mlsanity.com/paper/2301.01020/) (2023)
- [XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words](https://mlsanity.com/paper/2310.05235/) (2023)
- [Spoken Word2Vec: Learning Skipgram Embeddings from Speech](https://mlsanity.com/paper/2311.09319/) (2023)
- [Joint Multimodal Contrastive Learning for Robust Spoken Term Detection and Keyword Spotting](https://mlsanity.com/paper/2512.14115/) (2025)

## Abstract (arXiv)

We propose a new unsupervised model for mapping a variable-duration speech segment to a fixed-dimensional representation. The resulting acoustic word embeddings can form the basis of search, discovery, and indexing systems for low- and zero-resource languages. Our model, which we refer to as a maximal sampling correspondence variational autoencoder (MCVAE), is a recurrent neural network (RNN) trained with a novel self-supervised correspondence loss that encourages consistency between embeddings of different instances of the same word. Our training scheme improves on previous correspondence training approaches through the use and comparison of multiple samples from the approximate posterior distribution. In the zero-resource setting, the MCVAE can be trained in an unsupervised way, without any ground-truth word pairs, by using the word-like segments discovered via an unsupervised term discovery system. In both this setting and a semi-supervised low-resource setting (with a limited set of ground-truth word pairs), the MCVAE outperforms previous state-of-the-art models, such as Siamese-, CAE- and VAE-based RNNs.