# Fine-tune the pretrained ATST model for sound event detection

arXiv: https://arxiv.org/abs/2309.08153 · HTML: https://mlsanity.com/paper/2309.08153/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2023.

## Built on by (most cited first, 14 of 14)

- [Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning](https://mlsanity.com/paper/2512.19687/) (2025)
- [TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining](https://mlsanity.com/paper/2505.07609/) (2025)
- [AND: Audio Network Dissection for Interpreting Deep Acoustic Models](https://mlsanity.com/paper/2406.16990/) (2024)
- [MAT-SED: A Masked Audio Transformer with Masked-Reconstruction Based Pre-training for Sound Event Detection](https://mlsanity.com/paper/2408.08673/) (2024)
- [Language-based Audio Moment Retrieval](https://mlsanity.com/paper/2409.15672/) (2024)
- [Robust detection of overlapping bioacoustic sound events](https://mlsanity.com/paper/2503.02389/) (2025)
- [FlexSED: Towards Open-Vocabulary Sound Event Detection](https://mlsanity.com/paper/2509.18606/) (2025)
- [Prototype based Masked Audio Model for Self-Supervised Learning of Sound Event Detection](https://mlsanity.com/paper/2409.17656/) (2024)
- [MTDA-HSED: Mutual-Assistance Tuning and Dual-Branch Aggregating for Heterogeneous Sound Event Detection](https://mlsanity.com/paper/2409.06196/) (2024)
- [Exploring Performance-Complexity Trade-Offs in Sound Event Detection Models](https://mlsanity.com/paper/2503.11373/) (2025)
- [Formula-Supervised Sound Event Detection: Pre-Training Without Real Data](https://mlsanity.com/paper/2504.04428/) (2025)
- [Revisiting SSL for sound event detection: complementary fusion and adaptive post-processing](https://mlsanity.com/paper/2505.11889/) (2025)
- [Semi-Supervised Sound Event Detection with Conditional Mixup and Embedding-Level Contrastive Loss](https://mlsanity.com/paper/2606.29901/) (2026)
- [Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception](https://mlsanity.com/paper/2609.15215/) (2026)

## Abstract (arXiv)

Sound event detection (SED) often suffers from the data deficiency problem. The recent baseline system in the DCASE2023 challenge task 4 leverages the large pretrained self-supervised learning (SelfSL) models to mitigate such restriction, where the pretrained models help to produce more discriminative features for SED. However, the pretrained models are regarded as a frozen feature extractor in the challenge baseline system and most of the challenge submissions, and fine-tuning of the pretrained models has been rarely studied. In this work, we study the fine-tuning method of the pretrained models for SED. We first introduce ATST-Frame, our newly proposed SelfSL model, to the SED system. ATST-Frame was especially designed for learning frame-level representations of audio signals and obtained state-of-the-art (SOTA) performances on a series of downstream tasks. We then propose a fine-tuning method for ATST-Frame using both (in-domain) unlabelled and labelled SED data. Our experiments show that, the proposed method overcomes the overfitting problem when fine-tuning the large pretrained network, and our SED system obtains new SOTA results of 0.587/0.812 PSDS1/PSDS2 scores on the DCASE challenge task 4 dataset.