mlsanityData to 7 Oct 2026

MPN: Multimodal Parallel Network for Audio-Visual Event Localization

arXiv 2104.02971 ↗First posted 7 Apr 2021cs.CV cs.AI cs.MMcited by 9

Not on the map. Why?

Abstract (arXiv)

Audio-visual event localization aims to localize an event that is both audible and visible in the wild, which is a widespread audio-visual scene analysis task for unconstrained videos. To address this task, we propose a Multimodal Parallel Network (MPN), which can perceive global semantics and unmixed local information parallelly. Specifically, our MPN framework consists of a classification subnetwork to predict event categories and a localization subnetwork to predict event boundaries. The classification subnetwork is constructed by the Multimodal Co-attention Module (MCM) and obtains global contexts. The localization subnetwork consists of Multimodal Bottleneck Attention Module (MBAM), which is designed to extract fine-grained segment-level contents. Extensive experiments demonstrate that our framework achieves the state-of-the-art performance both in fully supervised and weakly supervised settings on the Audio-Visual Event (AVE) dataset.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 14 of its 21 references that are in the map. First posted on arXiv 7 Apr 2021.

Builds on

14 of 21 references.

References · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2016Layer Normalizationcited by 5,579
  4. 2017Squeeze-and-Excitation Networkscited by 4,763
  5. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  6. 2019Dynamic Convolution: Attention over Convolution Kernelscited by 355
  7. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  8. 2017Objects that Soundcited by 254
  9. 2016SoundNet: Learning Sound Representations from Unlabeled Videocited by 244
  10. 2016Lip Reading Sentences in the Wildcited by 226
  11. 2018Learning to Separate Object Sounds by Watching Unlabeled Videocited by 114
  12. 2019Dual-modality seq2seq network for audio-visual event localizationcited by 72
2 more
  1. 2017Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learningcited by 67
  2. 2020Look, Listen, and Attend: Co-Attention Network for Self-Supervised Audio-Visual Representation Learningcited by 63

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 2; 2023: 0; 2024: 2; 2025: 3.

9 papers.

  1. 2022Contrastive Positive Sample Propagation along the Audio-Visual Event Linecited by 50
  2. 2021MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsingcited by 39
  3. 2022AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separationcited by 24
  4. 2025Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperationcited by 16
  5. 2024Locality-aware Cross-modal Correspondence Learning for Dense Audio-Visual Events Localizationcited by 10
  6. 2021Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attentioncited by 5
  7. 2025Audio-visual Event Localization on Portrait Mode Short Videoscited by 3
  8. 2024STNet: Deep Audio-Visual Fusion Network for Robust Speaker Trackingcited by 1
  9. 2025DEL: Dense Event Localization for Multi-modal Audio-Visual Understandingcited by 1