# Effective Pre-Training of Audio Transformers for Sound Event Detection

arXiv: https://arxiv.org/abs/2409.09546 · HTML: https://mlsanity.com/paper/2409.09546/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2024.

## Built on by (most cited first, 12 of 12)

- [NonverbalTTS: A Public English Corpus of Text-Aligned Nonverbal Vocalizations with Emotion Annotations for Text-to-Speech](https://mlsanity.com/paper/2507.13155/) (2025)
- [MOSS-Audio Technical Report](https://mlsanity.com/paper/2606.01802/) (2026)
- [TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining](https://mlsanity.com/paper/2505.07609/) (2025)
- [Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries](https://mlsanity.com/paper/2507.16343/) (2025)
- [Not in Sync: Unveiling Temporal Bias in Audio Chat Models](https://mlsanity.com/paper/2510.12185/) (2025)
- [FlexSED: Towards Open-Vocabulary Sound Event Detection](https://mlsanity.com/paper/2509.18606/) (2025)
- [Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention](https://mlsanity.com/paper/2607.05978/) (2026)
- [RealDESED: A Real-World Domestic Sound Event Detection Benchmark](https://mlsanity.com/paper/2607.16736/) (2026)
- [Exploring Performance-Complexity Trade-Offs in Sound Event Detection Models](https://mlsanity.com/paper/2503.11373/) (2025)
- [Empowering Long-form Omni-modal Understanding with Robust Audio Perception](https://mlsanity.com/paper/2607.10299/) (2026)
- [PIVOT: Physics-Grounded Verification for AI-Generated Audio-Video Detection](https://mlsanity.com/paper/2609.15562/) (2026)
- [NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speech](https://mlsanity.com/paper/2609.31892/) (2026)

## Abstract (arXiv)

We propose a pre-training pipeline for audio spectrogram transformers for frame-level sound event detection tasks. On top of common pre-training steps, we add a meticulously designed training routine on AudioSet frame-level annotations. This includes a balanced sampler, aggressive data augmentation, and ensemble knowledge distillation. For five transformers, we obtain a substantial performance improvement over previously available checkpoints both on AudioSet frame-level predictions and on frame-level sound event detection downstream tasks, confirming our pipeline's effectiveness. We publish the resulting checkpoints that researchers can directly fine-tune to build high-performance models for sound event detection tasks.