mlsanityData to 14 Oct 2026

AQUA-Bench: Beyond Finding Answers to Knowing When There Are None in Audio Question Answering

arXiv 2601.12248 ↗First posted 18 Jan 2026eess.AS cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent advances in audio-aware large language models have shown strong performance on audio question answering. However, existing benchmarks mainly cover answerable questions and overlook the challenge of unanswerable ones, where no reliable answer can be inferred from the audio. Such cases are common in real-world settings, where questions may be misleading, ill-posed, or incompatible with the information. To address this gap, we present AQUA-Bench, a benchmark for Audio Question Unanswerability Assessment. It systematically evaluates three scenarios: Absent Answer Detection (the correct option is missing), Incompatible Answer Set Detection (choices are categorically mismatched with the question), and Incompatible Audio Question Detection (the question is irrelevant or lacks sufficient grounding in the audio). By assessing these cases, AQUA-Bench offers a rigorous measure of model reliability and promotes the development of audio-language systems that are more robust and trustworthy. Our experiments suggest that while models excel on standard answerable tasks, they often face notable challenges with unanswerable ones, pointing to a blind spot in current audio-language understanding.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 48 of its 60 references that are in the map. First posted on arXiv 18 Jan 2026.

Builds on

48 of 60 references.

References · 48

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2025Qwen2.5-Omni Technical Reportcited by 736
  5. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  6. 2024Qwen2-Audio Technical Reportcited by 560
  7. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  8. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  9. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  10. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 219
  11. 2023Listen, Think, and Understandcited by 209
  12. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
36 more
  1. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 133
  2. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  3. 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mixcited by 117
  4. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
  5. 2023Joint Audio and Speech Understandingcited by 100
  6. 2025Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Modelscited by 95
  7. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 90
  8. 2024Recent Advances in Speech Language Models: A Surveycited by 87
  9. 2023BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writingcited by 79
  10. 2024A Survey on Speech Large Language Models for Understandingcited by 71
  11. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  12. 2022Vocalsound: A Dataset for Improving Human Vocal Sounds Recognitioncited by 62
  13. 2023AudioChatLlama: Towards General-Purpose Speech Abilities for LLMscited by 59
  14. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 43
  15. 2024Towards audio language modeling -- an overviewcited by 40
  16. 2025Discrete Audio Tokens: More Than a Survey!cited by 38
  17. 2024BLSP-Emo: Towards Empathetic Large Speech-Language Modelscited by 36
  18. 2024Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Modelscited by 31
  19. 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 31
  20. 2024Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generationcited by 27
  21. 2025Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Modelscited by 27
  22. 2023UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Modelscited by 25
  23. 2025ParaS2S: Benchmarking and Aligning Spoken Language Models for Paralinguistic-aware Speech-to-Speech Interactioncited by 24
  24. 2024Building a Taiwanese Mandarin Spoken Language Model: A First Attemptcited by 23
  25. 2025AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMscited by 22
  26. 2024Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Modelscited by 21
  27. 2023AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Modelscited by 19
  28. 2024Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Modelscited by 13
  29. 2025Game-Time: Evaluating Temporal Dynamics in Spoken Language Modelscited by 13
  30. 2024SpeechPrompt: Prompting Speech Language Models for Speech Processing Taskscited by 12
  31. 2025Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samplescited by 11
  32. 2025EchoMind: An Interrelated Multi-level Benchmark for Evaluating Empathetic Speech Language Modelscited by 11
  33. 2025Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognitioncited by 9
  34. 2025CLIP-UP: CLIP-Based Unanswerable Problem Detection for Visual Question Answeringcited by 7
  35. 2025From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Datacited by 7
  36. 2023Toward Unsupervised Realistic Visual Question Answeringcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answeringcited by 8
  2. 2026Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Modelscited by 4
  3. 2026The Sound of Absence: Audio-Language Embedding Models Struggle with Negationcited by 1
  4. 2026Tracing Audio Grounding and Answer Selection in Audio LLMscited by 1
  5. 2026Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Modelscited by 0
  6. 2026Broadening Uncertainty Estimation for Audio Question Answering Across Methods, Formats, and Inputscited by 0