# AQUALLM: Audio Question Answering Data Generation Using Large Language Models

arXiv: https://arxiv.org/abs/2312.17343 · HTML: https://mlsanity.com/paper/2312.17343/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 16 references that are in the map.
- First posted on arXiv 28 Dec 2023.

## Builds on (most cited first, 12 of 12)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Learning to Answer Questions in Dynamic Audio-Visual Scenarios](https://mlsanity.com/paper/2203.14072/) (2022)
- [Training Question Answering Models From Synthetic Data](https://mlsanity.com/paper/2002.09599/) (2020)
- [Clotho-AQA: A Crowdsourced Dataset for Audio Question Answering](https://mlsanity.com/paper/2204.09634/) (2022)
- [All You May Need for VQA are Image Captions](https://mlsanity.com/paper/2205.01883/) (2022)
- [Temporal Reasoning via Audio Question Answering](https://mlsanity.com/paper/1911.09655/) (2019)
- [Improving Unsupervised Question Answering via Summarization-Informed Question Generation](https://mlsanity.com/paper/2109.07954/) (2021)
- [Multi-Scale Attention for Audio Question Answering](https://mlsanity.com/paper/2305.17993/) (2023)
- [QURIOUS: Question Generation Pretraining for Text Generation](https://mlsanity.com/paper/2004.11026/) (2020)
- [CLEAR: A Dataset for Compositional Language and Elementary Acoustic Reasoning](https://mlsanity.com/paper/1811.10561/) (2018)

## Built on by (most cited first, 4 of 4)

- [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024)
- [Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models](https://mlsanity.com/paper/2505.04921/) (2025)
- [LightHouse: A Survey of AGI Hallucination](https://mlsanity.com/paper/2401.06792/) (2024)
- [AURA Score: A Metric For Holistic Audio Question Answering Evaluation](https://mlsanity.com/paper/2510.04934/) (2025)

## Abstract (arXiv)

Audio Question Answering (AQA) constitutes a pivotal task in which machines analyze both audio signals and natural language questions to produce precise natural language answers. The significance of possessing high-quality, diverse, and extensive AQA datasets cannot be overstated when aiming for the precision of an AQA system. While there has been notable focus on developing accurate and efficient AQA models, the creation of high-quality, diverse, and extensive datasets for the specific task at hand has not garnered considerable attention. To address this challenge, this work makes several contributions. We introduce a scalable AQA data generation pipeline, denoted as the AQUALLM framework, which relies on Large Language Models (LLMs). This framework utilizes existing audio-caption annotations and incorporates state-of-the-art LLMs to generate expansive, high-quality AQA datasets. Additionally, we present three extensive and high-quality benchmark datasets for AQA, contributing significantly to the progression of AQA research. AQA models trained on the proposed datasets set superior benchmarks compared to the existing state-of-the-art. Moreover, models trained on our datasets demonstrate enhanced generalizability when compared to models trained using human-annotated AQA data. Code and datasets will be accessible on GitHub~footnoteurlhttps://github.com/swarupbehera/AQUALLM.