# Improving weakly supervised sound event detection with self-supervised auxiliary tasks

arXiv: https://arxiv.org/abs/2106.06858 · HTML: https://mlsanity.com/paper/2106.06858/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 12 Jun 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification](https://mlsanity.com/paper/1608.04363/) (2016)
- [A multi-device dataset for urban acoustic scene classification](https://mlsanity.com/paper/1807.09840/) (2018)
- [Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection](https://mlsanity.com/paper/1702.06286/) (2017)
- [General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline](https://mlsanity.com/paper/1807.09902/) (2018)
- [Adaptive pooling operators for weakly labeled sound event detection](https://mlsanity.com/paper/1804.10070/) (2018)
- [Large-scale weakly supervised audio classification using gated convolutional neural network](https://mlsanity.com/paper/1710.00343/) (2017)
- [Audio Event Detection using Weakly Labeled Data](https://mlsanity.com/paper/1605.02401/) (2016)
- [Sound Event Detection and Time-Frequency Segmentation from Weakly Labelled Data](https://mlsanity.com/paper/1804.04715/) (2018)
- [Single Channel Audio Source Separation using Convolutional Denoising Autoencoders](https://mlsanity.com/paper/1703.08019/) (2017)
- [Towards joint sound scene and polyphonic sound event recognition](https://mlsanity.com/paper/1904.10408/) (2019)
- [Sound Event Detection by Multitask Learning of Sound Events and Scenes with Soft Scene Labels](https://mlsanity.com/paper/2002.05848/) (2020)
- [Deep CNN Framework for Audio Event Recognition using Weakly Labeled Web Data](https://mlsanity.com/paper/1707.02530/) (2017)
- [Joint Analysis of Acoustic Events and Scenes Based on Multitask Learning](https://mlsanity.com/paper/1904.12146/) (2019)
- [Denoising without access to clean data using a partitioned autoencoder](https://mlsanity.com/paper/1509.05982/) (2015)
- [Label-efficient audio classification through multitask learning and self-supervision](https://mlsanity.com/paper/1910.12587/) (2019)

## Built on by (most cited first, 4 of 4)

- [Audio Retrieval with WavText5K and CLAP Training](https://mlsanity.com/paper/2209.14275/) (2022)
- [Couple Learning for semi-supervised sound event detection](https://mlsanity.com/paper/2110.05809/) (2021)
- [Adapting Task-Oriented Dialogue Models for Email Conversations](https://mlsanity.com/paper/2208.09439/) (2022)
- [Before the Warning Comes Too Late: Incremental Phone-Scam Detection from Speech](https://mlsanity.com/paper/2609.20223/) (2026)

## Abstract (arXiv)

While multitask and transfer learning has shown to improve the performance of neural networks in limited data settings, they require pretraining of the model on large datasets beforehand. In this paper, we focus on improving the performance of weakly supervised sound event detection in low data and noisy settings simultaneously without requiring any pretraining task. To that extent, we propose a shared encoder architecture with sound event detection as a primary task and an additional secondary decoder for a self-supervised auxiliary task. We empirically evaluate the proposed framework for weakly supervised sound event detection on a remix dataset of the DCASE 2019 task 1 acoustic scene data with DCASE 2018 Task 2 sounds event data under 0, 10 and 20 dB SNR. To ensure we retain the localisation information of multiple sound events, we propose a two-step attention pooling mechanism that provides a time-frequency localisation of multiple audio events in the clip. The proposed framework with two-step attention outperforms existing benchmark models by 22.3%, 12.8%, 5.9% on 0, 10 and 20 dB SNR respectively. We carry out an ablation study to determine the contribution of the auxiliary task and two-step attention pooling to the SED performance improvement.