# Deep convolutional acoustic word embeddings using word-pair side information

arXiv: https://arxiv.org/abs/1510.01032 · HTML: https://mlsanity.com/paper/1510.01032/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2015.

## Built on by (most cited first, 27 of 27)

- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Models](https://mlsanity.com/paper/2210.16043/) (2022)
- [Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation](https://mlsanity.com/paper/2103.10731/) (2021)
- [Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Heads](https://mlsanity.com/paper/2308.14456/) (2023)
- [A Computational Framework for Slang Generation](https://mlsanity.com/paper/2102.01826/) (2021)
- [Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables](https://mlsanity.com/paper/2308.03805/) (2023)
- [Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study](https://mlsanity.com/paper/2106.08686/) (2021)
- [How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddings](https://mlsanity.com/paper/2109.10179/) (2021)
- [On Compressing Sequences for Self-Supervised Speech Models](https://mlsanity.com/paper/2210.07189/) (2022)
- [Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource language](https://mlsanity.com/paper/2106.12834/) (2021)
- [CNN-based Spoken Term Detection and Localization without Dynamic Programming](https://mlsanity.com/paper/2103.05468/) (2021)
- [Understanding Shared Speech-Text Representations](https://mlsanity.com/paper/2304.14514/) (2023)
- [Spoken Term Detection and Relevance Score Estimation using Dot-Product of Pronunciation Embeddings](https://mlsanity.com/paper/2210.11895/) (2022)
- [Towards hate speech detection in low-resource languages: Comparing ASR to acoustic word embeddings on Wolof and Swahili](https://mlsanity.com/paper/2306.00410/) (2023)
- [AdaMS: Deep Metric Learning with Adaptive Margin and Adaptive Scale for Acoustic Word Discrimination](https://mlsanity.com/paper/2210.14564/) (2022)
- [Urban Region Profiling via A Multi-Graph Representation Learning Framework](https://mlsanity.com/paper/2202.02074/) (2022)
- [Analyzing the Representational Geometry of Acoustic Word Embeddings](https://mlsanity.com/paper/2301.03012/) (2023)
- [Integrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddings](https://mlsanity.com/paper/2209.06633/) (2022)
- [Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings](https://mlsanity.com/paper/2210.12857/) (2022)
- [Leveraging multilingual transfer for unsupervised semantic acoustic word embeddings](https://mlsanity.com/paper/2307.02083/) (2023)
- [Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network](https://mlsanity.com/paper/2207.14568/) (2022)
- [Consistency Based Weakly Self-Supervised Learning for Human Activity Recognition with Wearables](https://mlsanity.com/paper/2408.07282/) (2024)
- [BEST-STD: Bidirectional Mamba-Enhanced Speech Tokenization for Spoken Term Detection](https://mlsanity.com/paper/2411.14100/) (2024)
- [Deep-Learning Approach for Tissue Classification using Acoustic Waves during Ablation with an Er:YAG Laser (Updated)](https://mlsanity.com/paper/2406.14570/) (2024)
- [Cross-Lingual Query-by-Example Spoken Term Detection: A Transformer-Based Approach](https://mlsanity.com/paper/2410.04091/) (2024)
- [PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting](https://mlsanity.com/paper/2603.18023/) (2026)

## Abstract (arXiv)

Recent studies have been revisiting whole words as the basic modelling unit in speech recognition and query applications, instead of phonetic units. Such whole-word segmental systems rely on a function that maps a variable-length speech segment to a vector in a fixed-dimensional space; the resulting acoustic word embeddings need to allow for accurate discrimination between different word types, directly in the embedding space. We compare several old and new approaches in a word discrimination task. Our best approach uses side information in the form of known word pairs to train a Siamese convolutional neural network (CNN): a pair of tied networks that take two speech segments as input and produce their embeddings, trained with a hinge loss that separates same-word pairs and different-word pairs by some margin. A word classifier CNN performs similarly, but requires much stronger supervision. Both types of CNNs yield large improvements over the best previously published results on the word discrimination task.