# A Framework for the Robust Evaluation of Sound Event Detection

arXiv: https://arxiv.org/abs/1910.08440 · HTML: https://mlsanity.com/paper/1910.08440/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2019.

## Built on by (most cited first, 29 of 29)

- [Self-supervised Audio Teacher-Student Transformer for Both Clip-level and Frame-level Tasks](https://mlsanity.com/paper/2306.04186/) (2023)
- [Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning](https://mlsanity.com/paper/2512.19687/) (2025)
- [TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining](https://mlsanity.com/paper/2505.07609/) (2025)
- [Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries](https://mlsanity.com/paper/2507.16343/) (2025)
- [MAT-SED: A Masked Audio Transformer with Masked-Reconstruction Based Pre-training for Sound Event Detection](https://mlsanity.com/paper/2408.08673/) (2024)
- [Voice activity detection in the wild: A data-driven approach using teacher-student training](https://mlsanity.com/paper/2105.04065/) (2021)
- [The impact of non-target events in synthetic soundscapes for sound event detection](https://mlsanity.com/paper/2109.14061/) (2021)
- [The ACM Multimedia 2022 Computational Paralinguistics Challenge: Vocalisations, Stuttering, Activity, & Mosquitoes](https://mlsanity.com/paper/2205.06799/) (2022)
- [FlexSED: Towards Open-Vocabulary Sound Event Detection](https://mlsanity.com/paper/2509.18606/) (2025)
- [Segment-level Metric Learning for Few-shot Bioacoustic Event Detection](https://mlsanity.com/paper/2207.07773/) (2022)
- [Prototype based Masked Audio Model for Self-Supervised Learning of Sound Event Detection](https://mlsanity.com/paper/2409.17656/) (2024)
- [Self-training with noisy student model and semi-supervised loss function for dcase 2021 challenge task 4](https://mlsanity.com/paper/2107.02569/) (2021)
- [Dual Knowledge Distillation for Efficient Sound Event Detection](https://mlsanity.com/paper/2402.02781/) (2024)
- [Energy Consumption Trends in Sound Event Detection Systems](https://mlsanity.com/paper/2409.08763/) (2024)
- [A benchmark of state-of-the-art sound event detection systems evaluated on synthetic soundscapes](https://mlsanity.com/paper/2202.01487/) (2022)
- [Auditory Intelligence: Understanding the World Through Sound](https://mlsanity.com/paper/2508.07829/) (2025)
- [SynSonic: Augmenting Sound Event Detection through Text-to-Audio Diffusion ControlNet and Effective Sample Filtering](https://mlsanity.com/paper/2509.18603/) (2025)
- [Audio Attacks and Defenses against AED Systems -- A Practical Study](https://mlsanity.com/paper/2106.07428/) (2021)
- [Deep Learning-Based Acoustic Mosquito Detection in Noisy Conditions Using Trainable Kernels and Augmentations](https://mlsanity.com/paper/2207.13843/) (2022)
- [Multi-encoder attention-based architectures for sound recognition with partial visual assistance](https://mlsanity.com/paper/2209.12826/) (2022)
- [A Hybrid System of Sound Event Detection Transformer and Frame-wise Model for DCASE 2022 Task 4](https://mlsanity.com/paper/2210.09529/) (2022)
- [AGS: An Dataset and Taxonomy for Domestic Scene Sound Event Recognition](https://mlsanity.com/paper/2308.15726/) (2023)
- [Meta-Decomposition: Dynamic Segmentation Approach Selection in IoT-based Activity Recognition](https://mlsanity.com/paper/2404.11742/) (2024)
- [MTDA-HSED: Mutual-Assistance Tuning and Dual-Branch Aggregating for Heterogeneous Sound Event Detection](https://mlsanity.com/paper/2409.06196/) (2024)
- [Formula-Supervised Sound Event Detection: Pre-Training Without Real Data](https://mlsanity.com/paper/2504.04428/) (2025)
- [Revisiting SSL for sound event detection: complementary fusion and adaptive post-processing](https://mlsanity.com/paper/2505.11889/) (2025)
- [EZhouNet:A framework based on graph neural network and anchor interval for the respiratory sound event detection](https://mlsanity.com/paper/2509.01153/) (2025)
- [More Than A Shortcut: A Hyperbolic Approach To Early-Exit Networks](https://mlsanity.com/paper/2511.00641/) (2025)
- [Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding](https://mlsanity.com/paper/2607.04383/) (2026)

## Abstract (arXiv)

This work defines a new framework for performance evaluation of polyphonic sound event detection (SED) systems, which overcomes the limitations of the conventional collar-based event decisions, event F-scores and event error rates. The proposed framework introduces a definition of event detection that is more robust against labelling subjectivity. It also resorts to polyphonic receiver operating characteristic (ROC) curves to deliver more global insight into system performance than F1-scores, and proposes a reduction of these curves into a single polyphonic sound detection score (PSDS), which allows system comparison independently from operating points (OPs). The presented method also delivers better insight into data biases and classification stability across sound classes. Furthermore, it can be tuned to varying applications in order to match a variety of user experience requirements. The benefits of the proposed approach are demonstrated by re-evaluating the baseline and two of the top-performing systems from DCASE 2019 Task 4.