# End-to-End Open Vocabulary Keyword Search

arXiv: https://arxiv.org/abs/2108.10357 · HTML: https://mlsanity.com/paper/2108.10357/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 27 references that are in the map.
- First posted on arXiv 23 Aug 2021.

## Builds on (most cited first, 4 of 4)

- [End-to-End Streaming Keyword Spotting](https://mlsanity.com/paper/1812.02802/) (2018)
- [End-to-End ASR-free Keyword Search from Speech](https://mlsanity.com/paper/1701.04313/) (2017)
- [Improved acoustic word embeddings for zero-resource languages using multilingual transfer](https://mlsanity.com/paper/2006.02295/) (2020)
- [SpeechYOLO: Detection and Localization of Speech Objects](https://mlsanity.com/paper/1904.07704/) (2019)

## Built on by (most cited first, 6 of 6)

- [Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting](https://mlsanity.com/paper/2206.15400/) (2022)
- [Deep LSTM Spoken Term Detection using Wav2Vec 2.0 Recognizer](https://mlsanity.com/paper/2210.11885/) (2022)
- [Pretraining End-to-End Keyword Search with Automatically Discovered Acoustic Units](https://mlsanity.com/paper/2407.04652/) (2024)
- [Boosting Tail Neural Network for Realtime Custom Keyword Spotting](https://mlsanity.com/paper/2205.12933/) (2022)
- [End-to-End Open Vocabulary Keyword Search With Multilingual Neural Representations](https://mlsanity.com/paper/2308.08027/) (2023)
- [Written Term Detection Improves Spoken Term Detection](https://mlsanity.com/paper/2407.04601/) (2024)

## Abstract (arXiv)

Recently, neural approaches to spoken content retrieval have become popular. However, they tend to be restricted in their vocabulary or in their ability to deal with imbalanced test settings. These restrictions limit their applicability in keyword search, where the set of queries is not known beforehand, and where the system should return not just whether an utterance contains a query but the exact location of any such occurrences. In this work, we propose a model directly optimized for keyword search. The model takes a query and an utterance as input and returns a sequence of probabilities for each frame of the utterance of the query having occurred in that frame. Experiments show that the proposed model not only outperforms similar end-to-end models on a task where the ratio of positive and negative trials is artificially balanced, but it is also able to deal with the far more challenging task of keyword search with its inherent imbalance. Furthermore, using our system to rescore the outputs an LVCSR-based keyword search system leads to significant improvements on the latter.