Teacher-Guided Pseudo Supervision and Cross-Modal Alignment for Audio-Visual Video Parsing
Not on the map. Why?
Abstract (arXiv)
Weakly-supervised audio-visual video parsing (AVVP) seeks to detect audible, visible, and audio-visual events without temporal annotations. Previous work has emphasized refining global predictions through contrastive or collaborative learning, but neglected stable segment-level supervision and class-aware cross-modal alignment. To address this, we propose two strategies: (1) an exponential moving average (EMA)-guided pseudo supervision framework that generates reliable segment-level masks via adaptive thresholds or top-k selection, offering stable temporal guidance beyond video-level labels; and (2) a class-aware cross-modal agreement (CMA) loss that aligns audio and visual embeddings at reliable segment-class pairs, ensuring consistency across modalities while preserving temporal structure. Evaluations on LLP and UnAV-100 datasets shows that our method achieves state-of-the-art (SOTA) performance across multiple metrics.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 17 Sep 2025.
Builds on
13 of 21 references.
References · 13
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
- 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
- 2020Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsingcited by 146
- 2023Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baselinecited by 46
- 2022Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsingcited by 29
- 2023Audio-Visual Instance Segmentationcited by 21
- 2023Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parsercited by 18
- 2024Boosting Audio Visual Question Answering via Key Semantic-Aware Cuescited by 14
- 2024CoLeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio-Visual Video Parsingcited by 10
- 2023CM-PIE: Cross-modal perception for interactive-enhanced audio-visual video parsingcited by 9
1 more
Built on by
Citing papers per year: 2026: 1.
1 paper.