mlsanityData to 7 Oct 2026

MAT-SED: A Masked Audio Transformer with Masked-Reconstruction Based Pre-training for Sound Event Detection

arXiv 2408.08673 ↗First posted 16 Aug 2024cs.SD cs.AI eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

Sound event detection (SED) methods that leverage a large pre-trained Transformer encoder network have shown promising performance in recent DCASE challenges. However, they still rely on an RNN-based context network to model temporal dependencies, largely due to the scarcity of labeled data. In this work, we propose a pure Transformer-based SED model with masked-reconstruction based pre-training, termed MAT-SED. Specifically, a Transformer with relative positional encoding is first designed as the context network, pre-trained by the masked-reconstruction task on all available target data in a self-supervised way. Both the encoder and the context network are jointly fine-tuned in a semi-supervised manner. Furthermore, a global-local feature fusion strategy is proposed to enhance the localization capability. Evaluation of MAT-SED on DCASE2023 task4 surpasses state-of-the-art performance, achieving 0.587/0.896 PSDS1/PSDS2 respectively.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 16 Aug 2024.

Builds on

19 of 31 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  8. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  9. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  10. 2018Self-Attention with Relative Position Representationscited by 1,112
  11. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  12. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
7 more
  1. 2021Efficient Training of Audio Transformers with Patchoutcited by 222
  2. 2017Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detectioncited by 46
  3. 2018A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labelingcited by 37
  4. 2019A Framework for the Robust Evaluation of Sound Event Detectioncited by 28
  5. 2023Fine-tune the pretrained ATST model for sound event detectioncited by 14
  6. 2021FilterAugment: An Acoustic Environmental Data Augmentation Methodcited by 8
  7. 2023Semi-supervsied Learning-based Sound Event Detection using Freuqency Dynamic Convolution with Large Kernel Attention for DCASE Challenge 2023 Task 4cited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 2.

7 papers.

  1. 2025TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretrainingcited by 9
  2. 2024Prototype based Masked Audio Model for Self-Supervised Learning of Sound Event Detectioncited by 3
  3. 2026RealDESED: A Real-World Domestic Sound Event Detection Benchmarkcited by 1
  4. 2025Exploring Performance-Complexity Trade-Offs in Sound Event Detection Modelscited by 0
  5. 2025Revisiting SSL for sound event detection: complementary fusion and adaptive post-processingcited by 0
  6. 2025DeepCoT: Deep Continual Transformers for Real-Time Inference on Data Streamscited by 0
  7. 2026Semi-Supervised Sound Event Detection with Conditional Mixup and Embedding-Level Contrastive Losscited by 0