Solution for Temporal Sound Localisation Task of ECCV Second Perception Test Challenge 2024
Not on the map. Why?
Abstract (arXiv)
This report proposes an improved method for the Temporal Sound Localisation (TSL) task, which localizes and classifies the sound events occurring in the video according to a predefined set of sound classes. The champion solution from last year's first competition has explored the TSL by fusing audio and video modalities with the same weight. Considering the TSL task aims to localize sound events, we conduct relevant experiments that demonstrated the superiority of sound features (Section 3). Based on our findings, to enhance audio modality features, we employ various models to extract audio features, such as InterVideo, CaVMAE, and VideoMAE models. Our approach ranks first in the final test with a score of 0.4925.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 20 references that are in the map. First posted on arXiv 29 Sep 2024.
Builds on
10 of 20 references.
References · 10
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
- 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
- 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 454
- 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
- 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 378
- 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
- 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
- 2022Contrastive Audio-Visual Masked Autoencodercited by 142
- 2022TALLFormer: Temporal Action Localization with a Long-memory Transformercited by 78
- 2021Semi-Supervised Multi-Modal Multi-Instance Multi-Label Deep Network with Optimal Transportcited by 28
Built on by
Not cited yet.