# Multi-view Recurrent Neural Acoustic Word Embeddings

arXiv: https://arxiv.org/abs/1611.04496 · HTML: https://mlsanity.com/paper/1611.04496/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2016.

## Built on by (most cited first, 20 of 20)

- [What Do Self-Supervised Speech Models Know About Words?](https://mlsanity.com/paper/2307.00162/) (2023)
- [Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation](https://mlsanity.com/paper/2103.10731/) (2021)
- [Matching Latent Encoding for Audio-Text based Keyword Spotting](https://mlsanity.com/paper/2306.05245/) (2023)
- [Do Acoustic Word Embeddings Capture Phonological Similarity? An Empirical Study](https://mlsanity.com/paper/2106.08686/) (2021)
- [Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource language](https://mlsanity.com/paper/2106.12834/) (2021)
- [CNN-based Spoken Term Detection and Localization without Dynamic Programming](https://mlsanity.com/paper/2103.05468/) (2021)
- [Spoken Term Detection and Relevance Score Estimation using Dot-Product of Pronunciation Embeddings](https://mlsanity.com/paper/2210.11895/) (2022)
- [AdaMS: Deep Metric Learning with Adaptive Margin and Adaptive Scale for Acoustic Word Discrimination](https://mlsanity.com/paper/2210.14564/) (2022)
- [Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations](https://mlsanity.com/paper/2403.08738/) (2024)
- [Relational Proxy Loss for Audio-Text based Keyword Spotting](https://mlsanity.com/paper/2406.05314/) (2024)
- [Integrating Form and Meaning: A Multi-Task Learning Model for Acoustic Word Embeddings](https://mlsanity.com/paper/2209.06633/) (2022)
- [Text-Aware Adapter for Few-Shot Keyword Spotting](https://mlsanity.com/paper/2412.18142/) (2024)
- [Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network](https://mlsanity.com/paper/2207.14568/) (2022)
- [BEST-STD: Bidirectional Mamba-Enhanced Speech Tokenization for Spoken Term Detection](https://mlsanity.com/paper/2411.14100/) (2024)
- [Adversarial Deep Metric Learning for Cross-Modal Audio-Text Alignment in Open-Vocabulary Keyword Spotting](https://mlsanity.com/paper/2505.16735/) (2025)
- [Timestamped Embedding-Matching Acoustic-to-Word CTC ASR](https://mlsanity.com/paper/2306.11473/) (2023)
- [Speech representation learning: Learning bidirectional encoders with single-view, multi-view, and multi-task methods](https://mlsanity.com/paper/2308.00129/) (2023)
- [CTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting](https://mlsanity.com/paper/2406.07923/) (2024)
- [Joint Multimodal Contrastive Learning for Robust Spoken Term Detection and Keyword Spotting](https://mlsanity.com/paper/2512.14115/) (2025)
- [MATE: Matryoshka Audio-Text Embeddings for Open-Vocabulary Keyword Spotting](https://mlsanity.com/paper/2601.14012/) (2026)

## Abstract (arXiv)

Recent work has begun exploring neural acoustic word embeddings---fixed-dimensional vector representations of arbitrary-length speech segments corresponding to words. Such embeddings are applicable to speech retrieval and recognition tasks, where reasoning about whole words may make it possible to avoid ambiguous sub-word representations. The main idea is to map acoustic sequences to fixed-dimensional vectors such that examples of the same word are mapped to similar vectors, while different-word examples are mapped to very different vectors. In this work we take a multi-view approach to learning acoustic word embeddings, in which we jointly learn to embed acoustic sequences and their corresponding character sequences. We use deep bidirectional LSTM embedding models and multi-view contrastive losses. We study the effect of different loss variants, including fixed-margin and cost-sensitive losses. Our acoustic word embeddings improve over previous approaches for the task of word discrimination. We also present results on other tasks that are enabled by the multi-view approach, including cross-view word discrimination and word similarity.