# A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddings

arXiv: https://arxiv.org/abs/2012.07387 · HTML: https://mlsanity.com/paper/2012.07387/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 10 of 10)

- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Self-supervised representation learning from 12-lead ECG data](https://mlsanity.com/paper/2103.12676/) (2021)
- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Models](https://mlsanity.com/paper/2210.16043/) (2022)
- [Generative Spoken Language Model based on continuous word-sized audio tokens](https://mlsanity.com/paper/2310.05224/) (2023)
- [Speech Sequence Embeddings using Nearest Neighbors Contrastive Learning](https://mlsanity.com/paper/2204.05148/) (2022)
- [Unsupervised Automatic Speech Recognition: A Review](https://mlsanity.com/paper/2106.04897/) (2021)
- [Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Pooling](https://mlsanity.com/paper/2306.02153/) (2023)
- [Leveraging multilingual transfer for unsupervised semantic acoustic word embeddings](https://mlsanity.com/paper/2307.02083/) (2023)
- [Supervised Acoustic Embeddings And Their Transferability Across Languages](https://mlsanity.com/paper/2301.01020/) (2023)

## Abstract (arXiv)

Many speech processing tasks involve measuring the acoustic similarity between speech segments. Acoustic word embeddings (AWE) allow for efficient comparisons by mapping speech segments of arbitrary duration to fixed-dimensional vectors. For zero-resource speech processing, where unlabelled speech is the only available resource, some of the best AWE approaches rely on weak top-down constraints in the form of automatically discovered word-like segments. Rather than learning embeddings at the segment level, another line of zero-resource research has looked at representation learning at the short-time frame level. Recent approaches include self-supervised predictive coding and correspondence autoencoder (CAE) models. In this paper we consider whether these frame-level features are beneficial when used as inputs for training to an unsupervised AWE model. We compare frame-level features from contrastive predictive coding (CPC), autoregressive predictive coding and a CAE to conventional MFCCs. These are used as inputs to a recurrent CAE-based AWE model. In a word discrimination task on English and Xitsonga data, all three representation learning approaches outperform MFCCs, with CPC consistently showing the biggest improvement. In cross-lingual experiments we find that CPC features trained on English can also be transferred to Xitsonga.