mlsanityData to 14 Oct 2026

Multi-level Attention Fusion Network for Audio-visual Event Recognition

arXiv 2106.06736 ↗First posted 12 Jun 2021cs.CV cs.MMcited by 7

Not on the map. Why?

Abstract (arXiv)

Event classification is inherently sequential and multimodal. Therefore, deep neural models need to dynamically focus on the most relevant time window and/or modality of a video. In this study, we propose the Multi-level Attention Fusion network (MAFnet), an architecture that can dynamically fuse visual and audio information for event recognition. Inspired by prior studies in neuroscience, we couple both modalities at different levels of visual and audio paths. Furthermore, the network dynamically highlights a modality at a given time window relevant to classify events. Experimental results in AVE (Audio-Visual Event), UCF51, and Kinetics-Sounds datasets show that the approach can effectively improve the accuracy in audio-visual event classification. Code is available at: https://github.com/numediart/MAFnet

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 34 of its 56 references that are in the map. First posted on arXiv 12 Jun 2021.

Builds on

34 of 56 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  7. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  8. 2018SlowFast Networks for Video Recognitioncited by 1,867
  9. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  10. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  11. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  12. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
22 more
  1. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  2. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 484
  3. 2017Look, Listen and Learncited by 458
  4. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  5. 2014Recurrent Models of Visual Attentioncited by 349
  6. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  7. 2017Objects that Soundcited by 256
  8. 2016SoundNet: Learning Sound Representations from Unlabeled Videocited by 247
  9. 2015Beyond Short Snippets: Deep Networks for Video Classificationcited by 233
  10. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  11. 2017Modulating early visual processing by languagecited by 167
  12. 2020Audiovisual SlowFast Networks for Video Recognitioncited by 146
  13. 2017YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Videocited by 109
  14. 2015Compact Bilinear Poolingcited by 97
  15. 2019Dual-modality seq2seq network for audio-visual event localizationcited by 72
  16. 2017Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learningcited by 67
  17. 2016VideoLSTM Convolves, Attends and Flows for Action Recognitioncited by 60
  18. 2017Attention Clusters: Purely Attention Based Local Feature Integration for Video Classificationcited by 29
  19. 2017TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognitioncited by 18
  20. 2017Dynamic Layer Normalization for Adaptive Neural Acoustic Modeling in Speech Recognitioncited by 14
  21. 2018Modality Attention for End-to-End Audio-visual Speech Recognitioncited by 13
  22. 2019From Visual to Acoustic Question Answeringcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 2; 2025: 0; 2026: 1.

7 papers.

  1. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
  2. 2023Progressive Spatio-temporal Perception for Audio-Visual Question Answeringcited by 29
  3. 2023Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learningcited by 26
  4. 2022Deep soccer captioning with transformer: dataset, semantics-related losses, and multi-level evaluationcited by 4
  5. 2024Attend-Fusion: Efficient Audio-Visual Fusion for Video Classificationcited by 3
  6. 2024SaSR-Net: Source-Aware Semantic Representation Network for Enhancing Audio-Visual Question Answeringcited by 2
  7. 2026AV-Unified: A Unified Framework for Audio-visual Scene Understandingcited by 0