mlsanityData to 14 Oct 2026

I see what you hear: a vision-inspired method to localize words

arXiv 2210.13567 ↗First posted 24 Oct 2022cs.CV cs.LG cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper explores the possibility of using visual object detection techniques for word localization in speech data. Object detection has been thoroughly studied in the contemporary literature for visual data. Noting that an audio can be interpreted as a 1-dimensional image, object localization techniques can be fundamentally useful for word localization. Building upon this idea, we propose a lightweight solution for word detection and localization. We use bounding box regression for word localization, which enables our model to detect the occurrence, offset, and duration of keywords in a given audio stream. We experiment with LibriSpeech and train a model to localize 1000 words. Compared to existing work, our method reduces model size by 94%, and improves the F1 score by 6.5%.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 24 Oct 2022.

Builds on

15 of 21 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  3. 2018YOLOv3: An Incremental Improvementcited by 2,659
  4. 2019Objects as Pointscited by 1,175
  5. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  6. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
  7. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  8. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 99
  9. 2017Deep Residual Learning for Small-Footprint Keyword Spottingcited by 68
  10. 2021Broadcasted Residual Learning for Efficient Keyword Spottingcited by 48
  11. 2020Streaming keyword spotting on mobile devicescited by 47
  12. 2018End-to-End Streaming Keyword Spottingcited by 25
3 more
  1. 2021You Only Hear Once: A YOLO-like Algorithm for Audio Segmentation and Sound Event Detectioncited by 10
  2. 2019SpeechYOLO: Detection and Localization of Speech Objectscited by 9
  3. 2020Optimize what matters: Training DNN-HMM Keyword Spotting Model Using End Metriccited by 9

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Improving vision-inspired keyword spotting using dynamic module skipping in streaming conformer encodercited by 3