mlsanityData to 7 Oct 2026

LINK: Adaptive Modality Interaction for Audio-Visual Video Parsing

arXiv 2412.20872 ↗First posted 30 Dec 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Audio-visual video parsing focuses on classifying videos through weak labels while identifying events as either visible, audible, or both, alongside their respective temporal boundaries. Many methods ignore that different modalities often lack alignment, thereby introducing extra noise during modal interaction. In this work, we introduce a Learning Interaction method for Non-aligned Knowledge (LINK), designed to equilibrate the contributions of distinct modalities by dynamically adjusting their input during event prediction. Additionally, we leverage the semantic information of pseudo-labels as a priori knowledge to mitigate noise from other modalities. Our experimental findings demonstrate that our model outperforms existing methods on the LLP dataset.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 30 Dec 2024.

Builds on

19 of 26 references.

References · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  3. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  4. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 340
  5. 2022Balanced Multimodal Learning via On-the-fly Gradient Modulationcited by 259
  6. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  7. 2019Listen to Look: Action Recognition by Previewing Audiocited by 163
  8. 2020Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsingcited by 146
  9. 2023VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Datasetcited by 138
  10. 2024Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusioncited by 83
  11. 2021MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsingcited by 39
  12. 2023Language-driven All-in-one Adverse Weather Removalcited by 36
7 more
  1. 2023Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Taskscited by 31
  2. 2022Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsingcited by 29
  3. 2024Label-anticipated Event Disentanglement for Audio-Visual Video Parsingcited by 25
  4. 2023Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parsercited by 18
  5. 2024CoLeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio-Visual Video Parsingcited by 10
  6. 2022Scaling Multimodal Pre-Training via Cross-Modality Gradient Harmonizationcited by 9
  7. 2023CM-PIE: Cross-modal perception for interactive-enhanced audio-visual video parsingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025MUG: Pseudo Labeling Augmented Audio-Visual Mamba Network for Audio-Visual Video Parsingcited by 3
  2. 2025TEn-CATG:Text-Enriched Audio-Visual Video Parsing with Multi-Scale Category-Aware Temporal Graphcited by 1
  3. 2026EAR: Enhancing Uni-Modal Representations for Weakly Supervised Audio-Visual Video Parsingcited by 0