# End-to-End ASR-free Keyword Search from Speech

arXiv: https://arxiv.org/abs/1701.04313 · HTML: https://mlsanity.com/paper/1701.04313/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jan 2017.

## Built on by (most cited first, 19 of 19)

- [Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT](https://mlsanity.com/paper/2102.07594/) (2021)
- [Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Loss](https://mlsanity.com/paper/2102.07061/) (2021)
- [Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation](https://mlsanity.com/paper/2103.10731/) (2021)
- [MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition](https://mlsanity.com/paper/2303.05309/) (2023)
- [Visual Keyword Spotting with Attention](https://mlsanity.com/paper/2110.15957/) (2021)
- [Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource language](https://mlsanity.com/paper/2106.12834/) (2021)
- [End-to-End Open Vocabulary Keyword Search](https://mlsanity.com/paper/2108.10357/) (2021)
- [Speech-to-SQL: Towards Speech-driven SQL Query Generation From Natural Language Question](https://mlsanity.com/paper/2201.01209/) (2022)
- [Unsupervised Automatic Speech Recognition: A Review](https://mlsanity.com/paper/2106.04897/) (2021)
- [Towards hate speech detection in low-resource languages: Comparing ASR to acoustic word embeddings on Wolof and Swahili](https://mlsanity.com/paper/2306.00410/) (2023)
- [End-to-end LSTM based estimation of volcano event epicenter localization](https://mlsanity.com/paper/2110.14594/) (2021)
- [Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spotting](https://mlsanity.com/paper/2211.06478/) (2022)
- [Spot keywords from very noisy and mixed speech](https://mlsanity.com/paper/2305.17706/) (2023)
- [End-to-End User-Defined Keyword Spotting using Shifted Delta Coefficients](https://mlsanity.com/paper/2405.14489/) (2024)
- [Pretraining End-to-End Keyword Search with Automatically Discovered Acoustic Units](https://mlsanity.com/paper/2407.04652/) (2024)
- [Structural Segmentation and Labeling of Tabla Solo Performances](https://mlsanity.com/paper/2211.08790/) (2022)
- [End-to-End Open Vocabulary Keyword Search With Multilingual Neural Representations](https://mlsanity.com/paper/2308.08027/) (2023)
- [High-precision Voice Search Query Correction via Retrievable Speech-text Embedings](https://mlsanity.com/paper/2401.04235/) (2024)
- [Written Term Detection Improves Spoken Term Detection](https://mlsanity.com/paper/2407.04601/) (2024)

## Abstract (arXiv)

End-to-end (E2E) systems have achieved competitive results compared to conventional hybrid hidden Markov model (HMM)-deep neural network based automatic speech recognition (ASR) systems. Such E2E systems are attractive due to the lack of dependence on alignments between input acoustic and output grapheme or HMM state sequence during training. This paper explores the design of an ASR-free end-to-end system for text query-based keyword search (KWS) from speech trained with minimal supervision. Our E2E KWS system consists of three sub-systems. The first sub-system is a recurrent neural network (RNN)-based acoustic auto-encoder trained to reconstruct the audio through a finite-dimensional representation. The second sub-system is a character-level RNN language model using embeddings learned from a convolutional neural network. Since the acoustic and text query embeddings occupy different representation spaces, they are input to a third feed-forward neural network that predicts whether the query occurs in the acoustic utterance or not. This E2E ASR-free KWS system performs respectably despite lacking a conventional ASR system and trains much faster.