mlsanityData to 7 Oct 2026

TEn-CATG:Text-Enriched Audio-Visual Video Parsing with Multi-Scale Category-Aware Temporal Graph

arXiv 2509.04086 ↗First posted 4 Sep 2025cs.CV cs.MMcited by 1

Not on the map. Why?

Abstract (arXiv)

Audio-visual video parsing (AVVP) aims to detect event categories and their temporal boundaries in videos, typically under weak supervision. Existing methods mainly focus on (i) improving temporal modeling using attention-based architectures or (ii) generating richer pseudo-labels to address the absence of frame-level annotations. However, attention-based models often overfit noisy pseudo-labels, leading to cumulative training errors, while pseudo-label generation approaches distribute attention uniformly across frames, weakening temporal localization accuracy. To address these challenges, we propose TEn-CATG, a text-enriched AVVP framework that combines semantic calibration with category-aware temporal reasoning. More specifically, we design a bi-directional text fusion (BiT) module by leveraging audio-visual features as semantic anchors to refine text embeddings, which departs from conventional text-to-feature alignment, thereby mitigating noise and enhancing cross-modal consistency. Furthermore, we introduce the category-aware temporal graph (CATG) module to model temporal relationships by selecting multi-scale temporal neighbors and learning category-specific temporal decay factors, enabling effective event-dependent temporal reasoning. Extensive experiments demonstrate that TEn-CATG achieves state-of-the-art results across multiple evaluation metrics on benchmark datasets LLP and UnAV-100, highlighting its robustness and superior ability to capture complex temporal and semantic dependencies in weakly supervised AVVP tasks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

34 of 50 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2017Graph Attention Networkscited by 8,854
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  6. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  7. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  8. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  9. 2020Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsingcited by 146
  10. 2021Audio Retrieval with Natural Language Queries: A Benchmark Studycited by 61
  11. 2023Dense-Localizing Audio-Visual Events in Untrimmed Videos: A Large-Scale Benchmark and Baselinecited by 45
  12. 2021MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsingcited by 39
22 more
  1. 2022VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learningcited by 34
  2. 2023BAVS: Bootstrapping Audio-Visual Segmentation by Integrating Foundation Knowledgecited by 30
  3. 2022Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsingcited by 29
  4. 2022Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detectioncited by 29
  5. 2024Meerkat: Audio-Visual Large Language Model for Grounding in Space and Timecited by 26
  6. 2024Label-anticipated Event Disentanglement for Audio-Visual Video Parsingcited by 25
  7. 2023Audio-Visual Instance Segmentationcited by 21
  8. 2023HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video Understandingcited by 21
  9. 2023Language-Guided Audio-Visual Source Separation via Trimodal Consistencycited by 20
  10. 2024Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labelingcited by 20
  11. 2024T-VSL: Text-Guided Visual Sound Source Localization in Mixturescited by 19
  12. 2023Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parsercited by 18
  13. 2021HetEmotionNet: Two-Stream Heterogeneous Graph Recurrent Neural Network for Multi-modal Emotion Recognitioncited by 15
  14. 2023Sounding Video Generator: A Unified Framework for Text-guided Sounding Video Generationcited by 15
  15. 2022VoViT: Low Latency Graph-based Audio-Visual Voice Separation Transformercited by 14
  16. 2023Revisit Weakly-Supervised Audio-Visual Video Parsing from the Language Perspectivecited by 14
  17. 2024CoLeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio-Visual Video Parsingcited by 10
  18. 2023TMac: Temporal Multi-Modal Graph Learning for Acoustic Event Classificationcited by 9
  19. 2024Text-Guided Video Masked Autoencodercited by 8
  20. 2024LINK: Adaptive Modality Interaction for Audio-Visual Video Parsingcited by 3
  21. 2024Text-Queried Target Sound Event Localizationcited by 2
  22. 2025TeMTG: Text-Enhanced Multi-Hop Temporal Graph Modeling for Audio-Visual Video Parsingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025MUG: Pseudo Labeling Augmented Audio-Visual Mamba Network for Audio-Visual Video Parsing
  2. 2024Empowering LLMs with Pseudo-Untrimmed Videos for Audio-Visual Temporal Understanding
  3. 2025ESG-Net: Event-Aware Semantic Guided Network for Dense Audio-Visual Event Localization
  4. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding
  5. 2023Jointly Visual- and Semantic-Aware Graph Memory Networks for Temporal Sentence Localization in Videos
  6. 2023SViTT: Temporal Learning of Sparse Video-Text Transformers
  7. 2026ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting
  8. 2025AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestration