# You Only Hear Once: A YOLO-like Algorithm for Audio Segmentation and Sound Event Detection

arXiv: https://arxiv.org/abs/2109.00962 · HTML: https://mlsanity.com/paper/2109.00962/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 59 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks](https://mlsanity.com/paper/1312.6229/) (2013)
- [The Benefit Of Temporally-Strong Labels In Audio Event Classification](https://mlsanity.com/paper/2105.07031/) (2021)
- [Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection](https://mlsanity.com/paper/1702.06286/) (2017)
- [Sound Event Detection of Weakly Labelled Data with CNN-Transformer and Automatic Threshold Optimization](https://mlsanity.com/paper/1912.04761/) (2019)
- [Raw Waveform-based Audio Classification Using Sample-level CNN Architectures](https://mlsanity.com/paper/1712.00866/) (2017)
- [Towards duration robust weakly supervised sound event detection](https://mlsanity.com/paper/2101.07687/) (2021)
- [SpeechYOLO: Detection and Localization of Speech Objects](https://mlsanity.com/paper/1904.07704/) (2019)
- [Sound Event Detection and Separation: a Benchmark on Desed Synthetic Soundscapes](https://mlsanity.com/paper/2011.00801/) (2020)
- [A report on sound event detection with different binaural features](https://mlsanity.com/paper/1710.02997/) (2017)
- [Polyphonic Sound Event Detection by using Capsule Neural Networks](https://mlsanity.com/paper/1810.06325/) (2018)
- [Artificially Synthesising Data for Audio Classification and Segmentation to Improve Speech and Music Detection in Radio Broadcast](https://mlsanity.com/paper/2102.09959/) (2021)

## Built on by (most cited first, 10 of 10)

- [Unsupervised classification to improve the quality of a bird song recording dataset](https://mlsanity.com/paper/2302.07560/) (2023)
- [I see what you hear: a vision-inspired method to localize words](https://mlsanity.com/paper/2210.13567/) (2022)
- [A multi-head deep fusion model for recognition of cattle foraging events using sound and movement signals](https://mlsanity.com/paper/2505.10198/) (2025)
- [DEL: Dense Event Localization for Multi-modal Audio-Visual Understanding](https://mlsanity.com/paper/2506.23196/) (2025)
- [Evaluating robustness of You Only Hear Once(YOHO) Algorithm on noisy audios in the VOICe Dataset](https://mlsanity.com/paper/2111.01205/) (2021)
- [Human Voice Pitch Estimation: A Convolutional Network with Auto-Labeled and Synthetic Data](https://mlsanity.com/paper/2308.07170/) (2023)
- [EZhouNet:A framework based on graph neural network and anchor interval for the respiratory sound event detection](https://mlsanity.com/paper/2509.01153/) (2025)
- [Spectral and Rhythm Feature Performance Evaluation for Category and Class Level Audio Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2509.07756/) (2025)
- [A Neuromorphic Trigger for Efficient Audio Event Detection](https://mlsanity.com/paper/2606.17775/) (2026)
- [MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training](https://mlsanity.com/paper/2609.15221/) (2026)

## Abstract (arXiv)

Audio segmentation and sound event detection are crucial topics in machine listening that aim to detect acoustic classes and their respective boundaries. It is useful for audio-content analysis, speech recognition, audio-indexing, and music information retrieval. In recent years, most research articles adopt segmentation-by-classification. This technique divides audio into small frames and individually performs classification on these frames. In this paper, we present a novel approach called You Only Hear Once (YOHO), which is inspired by the YOLO algorithm popularly adopted in Computer Vision. We convert the detection of acoustic boundaries into a regression problem instead of frame-based classification. This is done by having separate output neurons to detect the presence of an audio class and predict its start and end points. The relative improvement for F-measure of YOHO, compared to the state-of-the-art Convolutional Recurrent Neural Network, ranged from 1% to 6% across multiple datasets for audio segmentation and sound event detection. As the output of YOHO is more end-to-end and has fewer neurons to predict, the speed of inference is at least 6 times faster than segmentation-by-classification. In addition, as this approach predicts acoustic boundaries directly, the post-processing and smoothing is about 7 times faster.