# Sound Event Detection of Weakly Labelled Data with CNN-Transformer and Automatic Threshold Optimization

arXiv: https://arxiv.org/abs/1912.04761 · HTML: https://mlsanity.com/paper/1912.04761/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2019.

## Built on by (most cited first, 29 of 29)

- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction](https://mlsanity.com/paper/2104.00563/) (2021)
- [Conditional Sound Generation Using Neural Discrete Time-Frequency Representation Learning](https://mlsanity.com/paper/2107.09998/) (2021)
- [Automated Audio Captioning: An Overview of Recent Progress and New Challenges](https://mlsanity.com/paper/2205.05949/) (2022)
- [Multiscale Audio Spectrogram Transformer for Efficient Audio Classification](https://mlsanity.com/paper/2303.10757/) (2023)
- [SepTr: Separable Transformer for Audio Spectrogram Processing](https://mlsanity.com/paper/2203.09581/) (2022)
- [CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification](https://mlsanity.com/paper/2203.06760/) (2022)
- [TranSPORTmer: A Holistic Approach to Trajectory Understanding in Multi-Agent Sports](https://mlsanity.com/paper/2410.17785/) (2024)
- [You Only Hear Once: A YOLO-like Algorithm for Audio Segmentation and Sound Event Detection](https://mlsanity.com/paper/2109.00962/) (2021)
- [Reliable Decision from Multiple Subtasks through Threshold Optimization: Content Moderation in the Wild](https://mlsanity.com/paper/2208.07522/) (2022)
- [From Weak to Strong Sound Event Labels using Adaptive Change-Point Detection and Active Learning](https://mlsanity.com/paper/2403.08525/) (2024)
- [Play It Back: Iterative Attention for Audio Recognition](https://mlsanity.com/paper/2210.11328/) (2022)
- [Towards Generating Diverse Audio Captions via Adversarial Training](https://mlsanity.com/paper/2212.02033/) (2022)
- [Fast threshold optimization for multi-label audio tagging using Surrogate gradient learning](https://mlsanity.com/paper/2103.00833/) (2021)
- [Cross-Referencing Self-Training Network for Sound Event Detection in Audio Mixtures](https://mlsanity.com/paper/2105.13392/) (2021)
- [Improving Weakly Supervised Sound Event Detection with Causal Intervention](https://mlsanity.com/paper/2303.05678/) (2023)
- [Exploring Transformers for On-Line Handwritten Signature Verification](https://mlsanity.com/paper/2307.01663/) (2023)
- [BAST: Binaural Audio Spectrogram Transformer for Binaural Sound Localization](https://mlsanity.com/paper/2207.03927/) (2022)
- [PrimaDNN': A Characteristics-aware DNN Customization for Singing Technique Detection](https://mlsanity.com/paper/2306.14191/) (2023)
- [AudioMosaic: Contrastive Masked Audio Representation Learning](https://mlsanity.com/paper/2605.14231/) (2026)
- [Adaptive Cost-Sensitive Learning in Neural Networks for Misclassification Cost Problems](https://mlsanity.com/paper/2111.07382/) (2021)
- [GAFX: A General Audio Feature eXtractor](https://mlsanity.com/paper/2207.09145/) (2022)
- [Improving Primate Sounds Classification using Binary Presorting for Deep Learning](https://mlsanity.com/paper/2306.16054/) (2023)
- [Asca: less audio data is more insightful](https://mlsanity.com/paper/2309.13373/) (2023)
- [Deformable Audio Transformer for Audio Event Detection](https://mlsanity.com/paper/2312.16228/) (2023)
- [SAM: A Mamba-2 State-Space Audio-Language Model](https://mlsanity.com/paper/2509.15680/) (2025)
- [Shared Representation Learning for Reference-Guided Targeted Sound Detection](https://mlsanity.com/paper/2603.17025/) (2026)
- [Acoustic UAV Detection in Battlefield Scenarios: Handling Noise, Domain Shift, and Weak Labels](https://mlsanity.com/paper/2608.14287/) (2026)
- [MAST: Label-Efficient, Robust, and Generalizable Sound Detection for Biodiversity Monitoring via Masked Audio Pretraining and Self-Training](https://mlsanity.com/paper/2609.15221/) (2026)

## Abstract (arXiv)

Sound event detection (SED) is a task to detect sound events in an audio recording. One challenge of the SED task is that many datasets such as the Detection and Classification of Acoustic Scenes and Events (DCASE) datasets are weakly labelled. That is, there are only audio tags for each audio clip without the onset and offset times of sound events. qkWe compare segment-wise and clip-wise training for SED that is lacking in previous works. We propose a convolutional neural network transformer (CNN-Transfomer) for audio tagging and SED, and show that CNN-Transformer performs similarly to a convolutional recurrent neural network (CRNN). Another challenge of SED is that thresholds are required for detecting sound events. Previous works set thresholds empirically, and are not an optimal approaches. To solve this problem, we propose an automatic threshold optimization method. The first stage is to optimize the system with respect to metrics that do not depend on thresholds, such as mean average precision (mAP). The second stage is to optimize the thresholds with respect to metrics that depends on those thresholds. Our proposed automatic threshold optimization system achieves a state-of-the-art audio tagging F1 of 0.646, outperforming that without threshold optimization of 0.629, and a sound event detection F1 of 0.584, outperforming that without threshold optimization of 0.564.