# SpeechYOLO: Detection and Localization of Speech Objects

arXiv: https://arxiv.org/abs/1904.07704 · HTML: https://mlsanity.com/paper/1904.07704/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2019.

## Built on by (most cited first, 9 of 9)

- [You Only Hear Once: A YOLO-like Algorithm for Audio Segmentation and Sound Event Detection](https://mlsanity.com/paper/2109.00962/) (2021)
- [CNN-based Spoken Term Detection and Localization without Dynamic Programming](https://mlsanity.com/paper/2103.05468/) (2021)
- [End-to-End Open Vocabulary Keyword Search](https://mlsanity.com/paper/2108.10357/) (2021)
- [Improving vision-inspired keyword spotting using dynamic module skipping in streaming conformer encoder](https://mlsanity.com/paper/2309.00140/) (2023)
- [The NPU System for the 2020 Personalized Voice Trigger Challenge](https://mlsanity.com/paper/2102.13552/) (2021)
- [I see what you hear: a vision-inspired method to localize words](https://mlsanity.com/paper/2210.13567/) (2022)
- [End-to-End Open Vocabulary Keyword Search With Multilingual Neural Representations](https://mlsanity.com/paper/2308.08027/) (2023)
- [Written Term Detection Improves Spoken Term Detection](https://mlsanity.com/paper/2407.04601/) (2024)
- [PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting](https://mlsanity.com/paper/2508.03190/) (2025)

## Abstract (arXiv)

In this paper, we propose to apply object detection methods from the vision domain on the speech recognition domain, by treating audio fragments as objects. More specifically, we present SpeechYOLO, which is inspired by the YOLO algorithm for object detection in images. The goal of SpeechYOLO is to localize boundaries of utterances within the input signal, and to correctly classify them. Our system is composed of a convolutional neural network, with a simple least-mean-squares loss function. We evaluated the system on several keyword spotting tasks, that include corpora of read speech and spontaneous speech. Our system compares favorably with other algorithms trained for both localization and classification.