mlsanityData to 14 Oct 2026

Multi-Scale Attention for Audio Question Answering

arXiv 2305.17993 ↗First posted 29 May 2023cs.SD cs.AI cs.MMcited by 13

Not on the map. Why?

Abstract (arXiv)

Audio question answering (AQA), acting as a widely used proxy task to explore scene understanding, has got more attention. The AQA is challenging for it requires comprehensive temporal reasoning from different scales' events of an audio scene. However, existing methods mostly extend the structures of visual question answering task to audio ones in a simple pattern but may not perform well when perceiving a fine-grained audio scene. To this end, we present a Multi-scale Window Attention Fusion Model (MWAFM) consisting of an asynchronous hybrid attention module and a multi-scale window attention module. The former is designed to aggregate unimodal and cross-modal temporal contexts, while the latter captures sound events of varying lengths and their temporal dependencies for a more comprehensive understanding. Extensive experiments are conducted to demonstrate that the proposed MWAFM can effectively explore temporal information to facilitate AQA in the fine-grained scene.Code: https://github.com/GeWu-Lab/MWAFM

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

19 of 25 references.

References · 19

  1. 2020Longformer: The Long-Document Transformercited by 3,413
  2. 2015VQA: Visual Question Answeringcited by 2,506
  3. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  4. 2019Clotho: An Audio Captioning Datasetcited by 358
  5. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  6. 2017Advances in Pre-Training Distributed Word Representationscited by 207
  7. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
  8. 2021Unsupervised Speech Recognitioncited by 148
  9. 2019Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answeringcited by 146
  10. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  11. 2022Clotho-AQA: A Crowdsourced Dataset for Audio Question Answeringcited by 85
  12. 2020An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separationcited by 81
7 more
  1. 2017Attention-Based Multimodal Fusion for Video Descriptioncited by 75
  2. 2021Sound Event Detection: A Tutorialcited by 51
  3. 2017Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detectioncited by 46
  4. 2019Temporal Reasoning via Audio Question Answeringcited by 39
  5. 2017Speech-Based Visual Question Answeringcited by 10
  6. 2018CLEAR: A Dataset for Compositional Language and Elementary Acoustic Reasoningcited by 7
  7. 2021NAAQA: A Neural Architecture for Acoustic Question Answeringcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 3; 2025: 3; 2026: 1.

13 papers.

Sort:

By area:

  1. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 317
  2. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 187
  3. 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Expertscited by 108
  4. 2023X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoningcited by 80
  5. 2023Auto-ACD: A Large-scale Dataset for Audio-Language Representation Learningcited by 33
  6. 2023Progressive Spatio-temporal Perception for Audio-Visual Question Answeringcited by 29
  7. 2023Object-aware Adaptive-Positivity Learning for Audio-Visual Question Answeringcited by 19
  8. 2025Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperationcited by 18
  9. 2025Learning Musical Representations for Music Performance Question Answeringcited by 17
  10. 2025Question-Aware Gaussian Experts for Audio-Visual Question Answeringcited by 12
  11. 2024Explore the Limits of Omni-modal Pretraining at Scalecited by 5
  12. 2023AQUALLM: Audio Question Answering Data Generation Using Large Language Modelscited by 4
1 more
  1. 2026AV-Unified: A Unified Framework for Audio-visual Scene Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AV-Master: Dual-Path Comprehensive Perception Makes Better Audio-Visual Question Answering
  2. 2024Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models
  3. 2024Answering Diverse Questions via Text Attached with Key Audio-Visual Clues
  4. 2024CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios
  5. 2024SHMamba: Structured Hyperbolic State Space Model for Audio-Visual Question Answering
  6. 2025Multi-Modal Scene Graph with Kolmogorov-Arnold Experts for Audio-Visual Question Answering
  7. 2025Multi-Domain Audio Question Answering Benchmark Toward Acoustic Content Reasoning
  8. 2025Audio-centric Video Understanding Benchmark without Text Shortcut