TEn-CATG:Text-Enriched Audio-Visual Video Parsing with Multi-Scale Category-Aware Temporal Graph
Not on the map. Why?
Abstract (arXiv)
Audio-visual video parsing (AVVP) aims to detect event categories and their temporal boundaries in videos, typically under weak supervision. Existing methods mainly focus on (i) improving temporal modeling using attention-based architectures or (ii) generating richer pseudo-labels to address the absence of frame-level annotations. However, attention-based models often overfit noisy pseudo-labels, leading to cumulative training errors, while pseudo-label generation approaches distribute attention uniformly across frames, weakening temporal localization accuracy. To address these challenges, we propose TEn-CATG, a text-enriched AVVP framework that combines semantic calibration with category-aware temporal reasoning. More specifically, we design a bi-directional text fusion (BiT) module by leveraging audio-visual features as semantic anchors to refine text embeddings, which departs from conventional text-to-feature alignment, thereby mitigating noise and enhancing cross-modal consistency. Furthermore, we introduce the category-aware temporal graph (CATG) module to model temporal relationships by selecting multi-scale temporal neighbors and learning category-specific temporal decay factors, enabling effective event-dependent temporal reasoning. Extensive experiments demonstrate that TEn-CATG achieves state-of-the-art results across multiple evaluation metrics on benchmark datasets LLP and UnAV-100, highlighting its robustness and superior ability to capture complex temporal and semantic dependencies in weakly supervised AVVP tasks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 4 Sep 2025.
Builds on
34 of 50 references.
References · 34
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2017Graph Attention Networkscited by 8,854
- 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
- 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
- 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
- 2020Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsingcited by 146
- 2021Audio Retrieval with Natural Language Queries: A Benchmark Studycited by 61
- 2023Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baselinecited by 45
- 2021MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsingcited by 39
22 more
- 2022VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learningcited by 34
- 2023BAVS: Bootstrapping Audio-Visual Segmentation by Integrating Foundation Knowledgecited by 30
- 2022Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsingcited by 29
- 2022Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detectioncited by 29
- 2024Meerkat: Audio-Visual Large Language Model for Grounding in Space and Timecited by 26
- 2024Label-anticipated Event Disentanglement for Audio-Visual Video Parsingcited by 25
- 2023Audio-Visual Instance Segmentationcited by 21
- 2023HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video Understandingcited by 21
- 2023Language-Guided Audio-Visual Source Separation via Trimodal Consistencycited by 20
- 2024Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labelingcited by 20
- 2024T-VSL: Text-Guided Visual Sound Source Localization in Mixturescited by 19
- 2023Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parsercited by 18
- 2021HetEmotionNet: Two-Stream Heterogeneous Graph Recurrent Neural Network for Multi-modal Emotion Recognitioncited by 15
- 2023Sounding Video Generator: A Unified Framework for Text-guided Sounding Video Generationcited by 15
- 2022VoViT: Low Latency Graph-based Audio-Visual Voice Separation Transformercited by 14
- 2023Revisit Weakly-Supervised Audio-Visual Video Parsing from the Language Perspectivecited by 14
- 2024CoLeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio-Visual Video Parsingcited by 10
- 2023TMac: Temporal Multi-Modal Graph Learning for Acoustic Event Classificationcited by 9
- 2024Text-Guided Video Masked Autoencodercited by 8
- 2024LINK: Adaptive Modality Interaction for Audio-Visual Video Parsingcited by 3
- 2024Text-Queried Target Sound Event Localizationcited by 2
- 2025TeMTG: Text-Enhanced Multi-Hop Temporal Graph Modeling for Audio-Visual Video Parsingcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025MUG: Pseudo Labeling Augmented Audio-Visual Mamba Network for Audio-Visual Video Parsing
- 2024Empowering LLMs with Pseudo-Untrimmed Videos for Audio-Visual Temporal Understanding
- 2025ESG-Net: Event-Aware Semantic Guided Network for Dense Audio-Visual Event Localization
- 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding
- 2023Jointly Visual- and Semantic-Aware Graph Memory Networks for Temporal Sentence Localization in Videos
- 2023SViTT: Temporal Learning of Sparse Video-Text Transformers
- 2026ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting
- 2025AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestration