mlsanityData to 14 Oct 2026

Revisiting SSL for sound event detection: complementary fusion and adaptive post-processing

arXiv 2505.11889 ↗First posted 17 May 2025eess.AS cs.AI cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) models offer powerful representations for sound event detection (SED), yet their synergistic potential remains underexplored. This study systematically evaluates state-of-the-art SSL models to guide optimal model selection and integration for SED. We propose a framework that combines heterogeneous SSL representations (e.g., BEATs, HuBERT, WavLM) through three fusion strategies: individual SSL embedding integration, dual-modal fusion, and full aggregation. Experiments on the DCASE 2023 Task 4 Challenge reveal that dual-modal fusion (e.g., CRNN+BEATs+WavLM) achieves complementary performance gains, while CRNN+BEATs alone delivers the best results among individual SSL models. We further introduce normalized sound event bounding boxes (nSEBBs), an adaptive post-processing method that dynamically adjusts event boundary predictions, improving PSDS1 by up to 4% for standalone SSL models. These findings highlight the compatibility and complementarity of SSL architectures, providing guidance for task-specific fusion and robust SED system design.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 44 references that are in the map. First posted on arXiv 17 May 2025.

Builds on

22 of 44 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  7. 2021AST: Audio Spectrogram Transformercited by 643
  8. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  9. 2015An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognitioncited by 520
  10. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 388
  11. 2022HEAR: Holistic Evaluation of Audio Representationscited by 78
  12. 2024Scaling up masked audio encoder learning for general audio classificationcited by 44
10 more
  1. 2019A Framework for the Robust Evaluation of Sound Event Detectioncited by 29
  2. 2023Self-supervised Audio Teacher-Student Transformer for Both Clip-level and Frame-level Taskscited by 25
  3. 2022Frequency Dynamic Convolution: Frequency-Adaptive Pattern Recognition for Sound Event Detectioncited by 23
  4. 2023Description and Discussion on DCASE 2023 Challenge Task 2: First-Shot Unsupervised Anomalous Sound Detection for Machine Condition Monitoringcited by 18
  5. 2023Fine-tune the pretrained ATST model for sound event detectioncited by 14
  6. 2024Sound Event Bounding Boxescited by 7
  7. 2024MAT-SED: A Masked Audio Transformer with Masked-Reconstruction Based Pre-training for Sound Event Detectioncited by 7
  8. 2023Semi-supervsied Learning-based Sound Event Detection using Freuqency Dynamic Convolution with Large Kernel Attention for DCASE Challenge 2023 Task 4cited by 4
  9. 2021Joint framework with deep feature distillation and adaptive focal loss for weakly supervised audio tagging and acoustic event detectioncited by 1
  10. 2024ICSD: An Open-source Dataset for Infant Cry and Snoring Detectioncited by 1

Built on by

Not cited yet.