# Temporal Reasoning via Audio Question Answering

arXiv: https://arxiv.org/abs/1911.09655 · HTML: https://mlsanity.com/paper/1911.09655/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2019.

## Built on by (most cited first, 30 of 39)

- [WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research](https://mlsanity.com/paper/2303.17395/) (2023)
- [Learning to Answer Questions in Dynamic Audio-Visual Scenarios](https://mlsanity.com/paper/2203.14072/) (2022)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videos](https://mlsanity.com/paper/2110.05122/) (2021)
- [Clotho-AQA: A Crowdsourced Dataset for Audio Question Answering](https://mlsanity.com/paper/2204.09634/) (2022)
- [CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios](https://mlsanity.com/paper/2403.04640/) (2024)
- [Progressive Spatio-temporal Perception for Audio-Visual Question Answering](https://mlsanity.com/paper/2308.05421/) (2023)
- [Automated Audio Captioning: An Overview of Recent Progress and New Challenges](https://mlsanity.com/paper/2205.05949/) (2022)
- [Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering](https://mlsanity.com/paper/2404.12020/) (2024)
- [Learning Musical Representations for Music Performance Question Answering](https://mlsanity.com/paper/2502.06710/) (2025)
- [Boosting Audio Visual Question Answering via Key Semantic-Aware Cues](https://mlsanity.com/paper/2407.20693/) (2024)
- [Multi-Scale Attention for Audio Question Answering](https://mlsanity.com/paper/2305.17993/) (2023)
- [Question-Aware Gaussian Experts for Audio-Visual Question Answering](https://mlsanity.com/paper/2503.04459/) (2025)
- [CAD -- Contextual Multi-modal Alignment for Dynamic AVQA](https://mlsanity.com/paper/2310.16754/) (2023)
- [TACOS: Temporally-aligned Audio CaptiOnS for Language-Audio Pretraining](https://mlsanity.com/paper/2505.07609/) (2025)
- [SHMamba: Structured Hyperbolic State Space Model for Audio-Visual Question Answering](https://mlsanity.com/paper/2406.09833/) (2024)
- [Evaluation of Question Answering Systems: Complexity of judging a natural language](https://mlsanity.com/paper/2209.12617/) (2022)
- [Answering Diverse Questions via Text Attached with Key Audio-Visual Clues](https://mlsanity.com/paper/2403.06679/) (2024)
- [Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models](https://mlsanity.com/paper/2409.06223/) (2024)
- [MAQA: A Multimodal QA Benchmark for Negation](https://mlsanity.com/paper/2301.03238/) (2023)
- [AQUALLM: Audio Question Answering Data Generation Using Large Language Models](https://mlsanity.com/paper/2312.17343/) (2023)
- [CLIP-Powered TASS: Target-Aware Single-Stream Network for Audio-Visual Question Answering](https://mlsanity.com/paper/2405.07451/) (2024)
- [Music Audio-Visual Question Answering Requires Specialized Multimodal Designs](https://mlsanity.com/paper/2505.20638/) (2025)
- [Learning Sparsity for Effective and Efficient Music Performance Question Answering](https://mlsanity.com/paper/2506.01319/) (2025)
- [NAAQA: A Neural Architecture for Acoustic Question Answering](https://mlsanity.com/paper/2106.06147/) (2021)
- [M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generation](https://mlsanity.com/paper/2402.11875/) (2024)
- [AV-Master: Dual-Path Comprehensive Perception Makes Better Audio-Visual Question Answering](https://mlsanity.com/paper/2510.18346/) (2025)
- [SaSR-Net: Source-Aware Semantic Representation Network for Enhancing Audio-Visual Question Answering](https://mlsanity.com/paper/2411.04933/) (2024)
- [Long-context Non-factoid Question Answering in Indic Languages](https://mlsanity.com/paper/2504.13615/) (2025)
- [Query-Guided Spatial-Temporal-Frequency Interaction for Music Audio-Visual Question Answering](https://mlsanity.com/paper/2601.19821/) (2026)

## Abstract (arXiv)

Multimodal question answering tasks can be used as proxy tasks to study systems that can perceive and reason about the world. Answering questions about different types of input modalities stresses different aspects of reasoning such as visual reasoning, reading comprehension, story understanding, or navigation. In this paper, we use the task of Audio Question Answering (AQA) to study the temporal reasoning abilities of machine learning models. To this end, we introduce the Diagnostic Audio Question Answering (DAQA) dataset comprising audio sequences of natural sound events and programmatically generated questions and answers that probe various aspects of temporal reasoning. We adapt several recent state-of-the-art methods for visual question answering to the AQA task, and use DAQA to demonstrate that they perform poorly on questions that require in-depth temporal reasoning. Finally, we propose a new model, Multiple Auxiliary Controllers for Linear Modulation (MALiMo) that extends the recent Feature-wise Linear Modulation (FiLM) model and significantly improves its temporal reasoning capabilities. We envisage DAQA to foster research on AQA and temporal reasoning and MALiMo a step towards models for AQA.