mlsanityData to 7 Oct 2026

End-to-End Open Vocabulary Keyword Search With Multilingual Neural Representations

arXiv 2308.08027 ↗First posted 15 Aug 2023eess.AS cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Conventional keyword search systems operate on automatic speech recognition (ASR) outputs, which causes them to have a complex indexing and search pipeline. This has led to interest in ASR-free approaches to simplify the search procedure. We recently proposed a neural ASR-free keyword search model which achieves competitive performance while maintaining an efficient and simplified pipeline, where queries and documents are encoded with a pair of recurrent neural network encoders and the encodings are combined with a dot-product. In this article, we extend this work with multilingual pretraining and detailed analysis of the model. Our experiments show that the proposed multilingual training significantly improves the model performance and that despite not matching a strong ASR-based conventional keyword search system for short queries and queries comprising in-vocabulary words, the proposed model outperforms the ASR-based system for long queries and queries that do not appear in the training data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 94 references that are in the map. First posted on arXiv 15 Aug 2023.

Builds on

25 of 94 references.

References · 25

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  3. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  4. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  5. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
  6. 2019Self-Training for End-to-End Speech Recognitioncited by 128
  7. 2022MAESTRO: Matched Speech Text Representations through Modality Matchingcited by 79
  8. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  9. 2019Generating Synthetic Audio Data for Attention-Based Speech Recognition Systemscited by 46
  10. 2020Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Trainingcited by 38
  11. 2022Integrating Text Inputs For Training and Adapting RNN Transducer ASR Modelscited by 20
  12. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
13 more
  1. 2017End-to-End ASR-free Keyword Search from Speechcited by 19
  2. 2020Multitask Training with Text Data for End-to-End Speech Recognitioncited by 19
  3. 2018Multi-Modal Data Augmentation for End-to-End ASRcited by 18
  4. 2020A Convolutional Deep Markov Model for Unsupervised Speech Representation Learningcited by 18
  5. 2021EAT: Enhanced ASR-TTS for Self-supervised Speech Recognitioncited by 13
  6. 2019Predicting detection filters for small footprint open-vocabulary keyword spottingcited by 10
  7. 2019SpeechYOLO: Detection and Localization of Speech Objectscited by 9
  8. 2019Neural Network based End-to-End Query by Example Spoken Term Detectioncited by 8
  9. 2022USTED: Improving ASR with a Unified Speech and Text Encoder-Decodercited by 7
  10. 2021CNN-based Spoken Term Detection and Localization without Dynamic Programmingcited by 6
  11. 2021End-to-End Open Vocabulary Keyword Searchcited by 6
  12. 2022Spoken Term Detection and Relevance Score Estimation using Dot-Product of Pronunciation Embeddingscited by 5
  13. 2022Deep LSTM Spoken Term Detection using Wav2Vec 2.0 Recognizercited by 3

Built on by

Not cited yet.