# AST-SED: An Effective Sound Event Detection Method Based on Audio Spectrogram Transformer

arXiv: https://arxiv.org/abs/2303.03689 · HTML: https://mlsanity.com/paper/2303.03689/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2023.

## Built on by (most cited first, 10 of 10)

- [Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learning](https://mlsanity.com/paper/2309.11500/) (2023)
- [Harnessing Vision Models for Time Series Analysis: A Survey](https://mlsanity.com/paper/2502.08869/) (2025)
- [Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries](https://mlsanity.com/paper/2507.16343/) (2025)
- [Language-based Audio Moment Retrieval](https://mlsanity.com/paper/2409.15672/) (2024)
- [Normalizing Energy Consumption for Hardware-Independent Evaluation](https://mlsanity.com/paper/2409.05602/) (2024)
- [Leveraging Language Model Capabilities for Sound Event Detection](https://mlsanity.com/paper/2308.11530/) (2023)
- [MTDA-HSED: Mutual-Assistance Tuning and Dual-Branch Aggregating for Heterogeneous Sound Event Detection](https://mlsanity.com/paper/2409.06196/) (2024)
- [Formula-Supervised Sound Event Detection: Pre-Training Without Real Data](https://mlsanity.com/paper/2504.04428/) (2025)
- [Towards Open World Sound Event Detection](https://mlsanity.com/paper/2605.03934/) (2026)
- [Semi-Supervised Sound Event Detection with Conditional Mixup and Embedding-Level Contrastive Loss](https://mlsanity.com/paper/2606.29901/) (2026)

## Abstract (arXiv)

In this paper, we propose an effective sound event detection (SED) method based on the audio spectrogram transformer (AST) model, pretrained on the large-scale AudioSet for audio tagging (AT) task, termed AST-SED. Pretrained AST models have recently shown promise on DCASE2022 challenge task4 where they help mitigate a lack of sufficient real annotated data. However, mainly due to differences between the AT and SED tasks, it is suboptimal to directly utilize outputs from a pretrained AST model. Hence the proposed AST-SED adopts an encoder-decoder architecture to enable effective and efficient fine-tuning without needing to redesign or retrain the AST model. Specifically, the Frequency-wise Transformer Encoder (FTE) consists of transformers with self attention along the frequency axis to address multiple overlapped audio events issue in a single clip. The Local Gated Recurrent Units Decoder (LGD) consists of nearest-neighbor interpolation (NNI) and Bidirectional Gated Recurrent Units (Bi-GRU) to compensate for temporal resolution loss in the pretrained AST model output. Experimental results on DCASE2022 task4 development set have demonstrated the superiority of the proposed AST-SED with FTE-LGD architecture. Specifically, the Event-Based F1-score (EB-F1) of 59.60% and Polyphonic Sound detection Score scenario1 (PSDS1) score of 0.5140 significantly outperform CRNN and other pretrained AST-based systems.