# Learning Joint Acoustic-Phonetic Word Embeddings

arXiv: https://arxiv.org/abs/1908.00493 · HTML: https://mlsanity.com/paper/1908.00493/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2019.

## Built on by (most cited first, 1 of 1)

- [PWESuite: Phonetic Word Embeddings and Tasks They Facilitate](https://mlsanity.com/paper/2304.02541/) (2023)

## Abstract (arXiv)

Most speech recognition tasks pertain to mapping words across two modalities: acoustic and orthographic. In this work, we suggest learning encoders that map variable-length, acoustic or phonetic, sequences that represent words into fixed-dimensional vectors in a shared latent space; such that the distance between two word vectors represents how closely the two words sound. Instead of directly learning the distances between word vectors, we employ weak supervision and model a binary classification task to predict whether two inputs, one of each modality, represent the same word given a distance threshold. We explore various deep-learning models, bimodal contrastive losses, and techniques for mining hard negative examples such as the semi-supervised technique of self-labeling. Our best model achieves an F1 score of 0.95 for the binary classification task.