mlsanityData to 7 Oct 2026

Dual-modality seq2seq network for audio-visual event localization

arXiv 1902.07473 ↗First posted 20 Feb 2019cs.CVcited by 72

Not on the map. Why?

Abstract (arXiv)

Audio-visual event localization requires one to identify theevent which is both visible and audible in a video (eitherat a frame or video level). To address this task, we pro-pose a deep neural network named Audio-Visual sequence-to-sequence dual network (AVSDN). By jointly taking bothaudio and visual features at each time segment as inputs, ourproposed model learns global and local event information ina sequence to sequence manner, which can be realized in ei-ther fully supervised or weakly supervised settings. Empiricalresults confirm that our proposed method performs favorablyagainst recent deep learning approaches in both settings.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 11; 2023: 20; 2024: 16; 2025: 7; 2026: 4.

72 papers.

Sort:

By area:

  1. 2022Audio-Visual Segmentationcited by 148
  2. 2022PMR: Prototypical Modal Rebalance for Multimodal Learningcited by 114
  3. 2023Audio-Visual Segmentation with Semanticscited by 73
  4. 2022Vision Transformers are Parameter-Efficient Audio-Visual Learnerscited by 72
  5. 2021Positive Sample Propagation along the Audio-Visual Event Linecited by 69
  6. 2023AVSegFormer: Audio-Visual Segmentation with Transformercited by 66
  7. 2021Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separationcited by 59
  8. 2023Audio-Visual Grouping Network for Sound Localization from Mixturescited by 52
  9. 2023CATR: Combinatorial-Dependence Audio-Queried Transformer for Audio-Visual Video Segmentationcited by 52
  10. 2022Contrastive Positive Sample Propagation along the Audio-Visual Event Linecited by 50
  11. 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspectivecited by 44
  12. 2023Annotation-free Audio-Visual Segmentationcited by 44
38 more
  1. 2022Exploiting Transformation Invariance and Equivariance for Self-supervised Sound Localisationcited by 43
  2. 2023Egocentric Audio-Visual Object Localizationcited by 40
  3. 2021MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsingcited by 39
  4. 2021Audio-Visual Synchronisation in the wildcited by 39
  5. 2022ECLIPSE: Efficient Long-range Video Retrieval using Sight and Soundcited by 39
  6. 2023Multimodal Variational Auto-encoder based Audio-Visual Segmentationcited by 37
  7. 2023Audio-Visual Class-Incremental Learningcited by 33
  8. 2024On-the-fly Modulation for Balanced Multimodal Learningcited by 32
  9. 2023Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Taskscited by 31
  10. 2022Joint-Modal Label Denoising for Weakly-Supervised Audio-Visual Video Parsingcited by 29
  11. 2021Unsupervised Sound Localization via Iterative Contrastive Learningcited by 28
  12. 2024Text-to-Audio Generation Synchronized with Videoscited by 28
  13. 2024Audio-Visual Segmentation via Unlabeled Frame Exploitationcited by 27
  14. 2024Meerkat: Audio-Visual Large Language Model for Grounding in Space and Timecited by 26
  15. 2024Towards Open-Vocabulary Audio-Visual Event Localizationcited by 26
  16. 2023Unveiling the Power of Audio-Visual Early Fusion Transformers with Dense Interactions through Masked Modelingcited by 24
  17. 2021Can audio-visual integration strengthen robustness under multimodal attacks?cited by 23
  18. 2023A Unified Audio-Visual Learning Framework for Localization, Separation, and Recognitioncited by 21
  19. 2024Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labelingcited by 20
  20. 2023Improving Audio-Visual Video Parsing with Pseudo Visual Labelscited by 18
  21. 2023Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parsercited by 18
  22. 2021Learning Audio-Visual Correlations from Variational Cross-Modal Generationcited by 14
  23. 2021Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generationcited by 14
  24. 2023Revisit Weakly-Supervised Audio-Visual Video Parsing from the Language Perspectivecited by 14
  25. 2025Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigationcited by 12
  26. 2023Multimodal Imbalance-Aware Gradient Modulation for Weakly-supervised Audio-Visual Video Parsingcited by 11
  27. 2023CAD -- Contextual Multi-modal Alignment for Dynamic AVQAcited by 10
  28. 2024CoLeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio-Visual Video Parsingcited by 10
  29. 2024NarrativeBridge: Enhancing Video Captioning with Causal-Temporal Narrativecited by 10
  30. 2024CACE-Net: Co-guidance Attention and Contrastive Enhancement for Effective Audio-Visual Event Localizationcited by 10
  31. 2024Locality-aware Cross-modal Correspondence Learning for Dense Audio-Visual Events Localizationcited by 10
  32. 2025Mettle: Meta-Token Learning for Memory-Efficient Audio-Visual Adaptationcited by 10
  33. 2021MPN: Multimodal Parallel Network for Audio-Visual Event Localizationcited by 9
  34. 2023CM-PIE: Cross-modal perception for interactive-enhanced audio-visual video parsingcited by 9
  35. 2025VGGSounder: Audio-Visual Evaluations for Foundation Modelscited by 8
  36. 2021Multi-level Attention Fusion Network for Audio-visual Event Recognitioncited by 7
  37. 2022Leveraging the Video-level Semantic Consistency of Event for Audio-visual Event Localizationcited by 7
  38. 2024Audio-visual Generalized Zero-shot Learning the Easy Waycited by 7

50 most cited of 72 All 72 (JSON)