# Sound Event Detection and Time-Frequency Segmentation from Weakly Labelled Data

arXiv: https://arxiv.org/abs/1804.04715 · HTML: https://mlsanity.com/paper/1804.04715/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2018.

## Built on by (most cited first, 10 of 10)

- [Automated Audio Captioning: An Overview of Recent Progress and New Challenges](https://mlsanity.com/paper/2205.05949/) (2022)
- [Robust Time Series Denoising with Learnable Wavelet Packet Transform](https://mlsanity.com/paper/2206.06126/) (2022)
- [Anomalous Sound Detection with Machine Learning: A Systematic Review](https://mlsanity.com/paper/2102.07820/) (2021)
- [Voice activity detection in the wild: A data-driven approach using teacher-student training](https://mlsanity.com/paper/2105.04065/) (2021)
- [Guided Masked Self-Distillation Modeling for Distributed Multimedia Sensor Event Analysis](https://mlsanity.com/paper/2404.08264/) (2024)
- [Improving weakly supervised sound event detection with self-supervised auxiliary tasks](https://mlsanity.com/paper/2106.06858/) (2021)
- [Audio-Visual Fusion Layers for Event Type Aware Video Recognition](https://mlsanity.com/paper/2202.05961/) (2022)
- [DeepExtractor: Time-domain reconstruction of signals and glitches in gravitational wave data with deep learning](https://mlsanity.com/paper/2501.18423/) (2025)
- [BGM2Pose: Active 3D Human Pose Estimation with Non-Stationary Sounds](https://mlsanity.com/paper/2503.00389/) (2025)
- [Multi-View Video-Based Learning: Leveraging Weak Labels for Frame-Level Perception](https://mlsanity.com/paper/2403.11616/) (2024)

## Abstract (arXiv)

Sound event detection (SED) aims to detect when and recognize what sound events happen in an audio clip. Many supervised SED algorithms rely on strongly labelled data which contains the onset and offset annotations of sound events. However, many audio tagging datasets are weakly labelled, that is, only the presence of the sound events is known, without knowing their onset and offset annotations. In this paper, we propose a time-frequency (T-F) segmentation framework trained on weakly labelled data to tackle the sound event detection and separation problem. In training, a segmentation mapping is applied on a T-F representation, such as log mel spectrogram of an audio clip to obtain T-F segmentation masks of sound events. The T-F segmentation masks can be used for separating the sound events from the background scenes in the time-frequency domain. Then a classification mapping is applied on the T-F segmentation masks to estimate the presence probabilities of the sound events. We model the segmentation mapping using a convolutional neural network and the classification mapping using a global weighted rank pooling (GWRP). In SED, predicted onset and offset times can be obtained from the T-F segmentation masks. As a byproduct, separated waveforms of sound events can be obtained from the T-F segmentation masks. We remixed the DCASE 2018 Task 1 acoustic scene data with the DCASE 2018 Task 2 sound events data. When mixing under 0 dB, the proposed method achieved F1 scores of 0.534, 0.398 and 0.167 in audio tagging, frame-wise SED and event-wise SED, outperforming the fully connected deep neural network baseline of 0.331, 0.237 and 0.120, respectively. In T-F segmentation, we achieved an F1 score of 0.218, where previous methods were not able to do T-F segmentation.