mlsanityData to 13 Oct 2026

SCoPE: Training-Free Audio-Visual Event Perception via Sparse Cross-Modal Prior Exchange

arXiv 2608.07923 ↗First posted 8 Aug 2026cs.CV cs.MM cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Audio-visual event perception (AVEP) determines which events occur in a video, when they occur, and whether they are audible, visible, or both. Training-free methods query new event vocabularies by matching frozen audio and visual features with text-encoded event names. However, related labels share evidence. An incorrect label can then score at least as high as a correct one. We call this a false co-activation (FCA). No scalar cutoff can reject the incorrect label while keeping every correct one. Class-specific thresholds may prevent that label from becoming a final prediction, but the FCA remains in the underlying score vector. We introduce SCoPE, a training-free framework in which all queried labels compete for shared evidence and each modality guides event selection in the other. We derive an exact condition for when this competition removes an FCA in a two-label fit. With identical frozen CLIP+CLAP backbones on LLP, SCoPE improves Type@seg by 7.45 points and Event@seg by 5.04 points compared with the reported AV2A values. The same fixed configuration transfers unchanged to OV-AVEBench and VGGSound-AVEL100k.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 8 Aug 2026.

Builds on

19 of 29 references.

References · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,168
  3. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 644
  4. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 555
  5. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 369
  6. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 206
  7. 2020Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsingcited by 146
  8. 2024Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)cited by 119
  9. 2020Semantic Audio-Visual Navigationcited by 79
  10. 2019Self-supervised Moving Vehicle Tracking with Stereo Soundcited by 63
  11. 2022Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsingcited by 29
  12. 2024Towards Open-Vocabulary Audio-Visual Event Localizationcited by 26
7 more
  1. 2024Label-anticipated Event Disentanglement for Audio-Visual Video Parsingcited by 25
  2. 2023Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parsercited by 18
  3. 2023Revisit Weakly-Supervised Audio-Visual Video Parsing from the Language Perspectivecited by 14
  4. 2025Transformation of audio embeddings into interpretable, concept-based representationscited by 5
  5. 2025PreFM: Online Audio-Visual Event Parsing via Predictive Future Modelingcited by 4
  6. 2025Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholdscited by 2
  7. 2026Hierarchical Semantic-Constrained Heterogeneous Graph for Audio-Visual Event Localizationcited by 1

Built on by

Not cited yet.