mlsanityData to 7 Oct 2026

Written Term Detection Improves Spoken Term Detection

arXiv 2407.04601 ↗First posted 5 Jul 2024eess.AS cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

End-to-end (E2E) approaches to keyword search (KWS) are considerably simpler in terms of training and indexing complexity when compared to approaches which use the output of automatic speech recognition (ASR) systems. This simplification however has drawbacks due to the loss of modularity. In particular, where ASR-based KWS systems can benefit from external unpaired text via a language model, current formulations of E2E KWS systems have no such mechanism. Therefore, in this paper, we propose a multitask training objective which allows unpaired text to be integrated into E2E KWS without complicating indexing and search. In addition to training an E2E KWS model to retrieve text queries from spoken documents, we jointly train it to retrieve text queries from masked written documents. We show empirically that this approach can effectively leverage unpaired text for KWS, with significant improvements in search performance across a wide variety of languages. We conduct analysis which indicates that these improvements are achieved because the proposed method improves document representations for words in the unpaired text. Finally, we show that the proposed method can be used for domain adaptation in settings where in-domain paired data is scarce or nonexistent.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 94 references that are in the map. First posted on arXiv 5 Jul 2024.

Builds on

25 of 94 references.

References · 25

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  3. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  4. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  5. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
  6. 2019Self-Training for End-to-End Speech Recognitioncited by 128
  7. 2022MAESTRO: Matched Speech Text Representations through Modality Matchingcited by 79
  8. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  9. 2019Generating Synthetic Audio Data for Attention-Based Speech Recognition Systemscited by 46
  10. 2020Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Trainingcited by 38
  11. 2022Integrating Text Inputs For Training and Adapting RNN Transducer ASR Modelscited by 20
  12. 2022Towards Reducing the Need for Speech Training Data To Build Spoken Language Understanding Systemscited by 20
13 more
  1. 2017End-to-End ASR-free Keyword Search from Speechcited by 19
  2. 2020Multitask Training with Text Data for End-to-End Speech Recognitioncited by 19
  3. 2018Multi-Modal Data Augmentation for End-to-End ASRcited by 18
  4. 2020A Convolutional Deep Markov Model for Unsupervised Speech Representation Learningcited by 18
  5. 2021EAT: Enhanced ASR-TTS for Self-supervised Speech Recognitioncited by 13
  6. 2019Predicting detection filters for small footprint open-vocabulary keyword spottingcited by 10
  7. 2019SpeechYOLO: Detection and Localization of Speech Objectscited by 9
  8. 2019Neural Network based End-to-End Query by Example Spoken Term Detectioncited by 8
  9. 2022USTED: Improving ASR with a Unified Speech and Text Encoder-Decodercited by 7
  10. 2021CNN-based Spoken Term Detection and Localization without Dynamic Programmingcited by 6
  11. 2021End-to-End Open Vocabulary Keyword Searchcited by 6
  12. 2022Spoken Term Detection and Relevance Score Estimation using Dot-Product of Pronunciation Embeddingscited by 5
  13. 2022Deep LSTM Spoken Term Detection using Wav2Vec 2.0 Recognizercited by 3

Built on by

Not cited yet.