# Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection

arXiv: https://arxiv.org/abs/2110.02011 · HTML: https://mlsanity.com/paper/2110.02011/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 52 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection](https://mlsanity.com/paper/1702.06286/) (2017)
- [A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling](https://mlsanity.com/paper/1810.09050/) (2018)
- [Adaptive pooling operators for weakly labeled sound event detection](https://mlsanity.com/paper/1804.10070/) (2018)
- [Sound Event Detection Using Spatial Features and Convolutional Recurrent Neural Network](https://mlsanity.com/paper/1706.02291/) (2017)
- [Large-Scale Weakly Labeled Semi-Supervised Sound Event Detection in Domestic Environments](https://mlsanity.com/paper/1807.10501/) (2018)
- [Guided Learning Convolution System for DCASE 2019 Task 4](https://mlsanity.com/paper/1909.06178/) (2019)
- [Sound event detection via dilated convolutional recurrent neural networks](https://mlsanity.com/paper/1911.10888/) (2019)
- [SONYC: A System for the Monitoring, Analysis and Mitigation of Urban Noise Pollution](https://mlsanity.com/paper/1805.00889/) (2018)
- [Specialized Decision Surface and Disentangled Feature for Weakly-Supervised Polyphonic Sound Event Detection](https://mlsanity.com/paper/1905.10091/) (2019)
- [Evaluation of post-processing algorithms for polyphonic sound event detection](https://mlsanity.com/paper/1906.06909/) (2019)
- [Guided learning for weakly-labeled semi-supervised sound event detection](https://mlsanity.com/paper/1906.02517/) (2019)
- [Robust Audio Event Recognition with 1-Max Pooling Convolutional Neural Networks](https://mlsanity.com/paper/1604.06338/) (2016)
- [Multi-Branch Learning for Weakly-Labeled Sound Event Detection](https://mlsanity.com/paper/2002.09661/) (2020)
- [DETR for Crowd Pedestrian Detection](https://mlsanity.com/paper/2012.06785/) (2020)
- [Weakly Labeled Sound Event Detection Using Tri-training and Adversarial Learning](https://mlsanity.com/paper/1910.06790/) (2019)
- [Learning generic feature representation with synthetic data for weakly-supervised sound event detection by inter-frame distance loss](https://mlsanity.com/paper/2011.00695/) (2020)

## Built on by (most cited first, 7 of 7)

- [SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models](https://mlsanity.com/paper/2508.02018/) (2025)
- [Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries](https://mlsanity.com/paper/2507.16343/) (2025)
- [DiffSED: Sound Event Detection with Denoising Diffusion](https://mlsanity.com/paper/2308.07293/) (2023)
- [Robust detection of overlapping bioacoustic sound events](https://mlsanity.com/paper/2503.02389/) (2025)
- [A Hybrid System of Sound Event Detection Transformer and Frame-wise Model for DCASE 2022 Task 4](https://mlsanity.com/paper/2210.09529/) (2022)
- [Towards Open World Sound Event Detection](https://mlsanity.com/paper/2605.03934/) (2026)
- [A Neuromorphic Trigger for Efficient Audio Event Detection](https://mlsanity.com/paper/2606.17775/) (2026)

## Abstract (arXiv)

Sound event detection (SED) has gained increasing attention with its wide application in surveillance, video indexing, etc. Existing models in SED mainly generate frame-level prediction, converting it into a sequence multi-label classification problem. A critical issue with the frame-based model is that it pursues the best frame-level prediction rather than the best event-level prediction. Besides, it needs post-processing and cannot be trained in an end-to-end way. This paper firstly presents the one-dimensional Detection Transformer (1D-DETR), inspired by Detection Transformer for image object detection. Furthermore, given the characteristics of SED, the audio query branch and a one-to-many matching strategy for fine-tuning the model are added to 1D-DETR to form Sound Event Detection Transformer (SEDT). To our knowledge, SEDT is the first event-based and end-to-end SED model. Experiments are conducted on the URBAN-SED dataset and the DCASE2019 Task4 dataset, and both show that SEDT can achieve competitive performance.