# AuditoryBench++: Can Language Models Understand Auditory Knowledge without Hearing?

arXiv: https://arxiv.org/abs/2509.17641 · HTML: https://mlsanity.com/paper/2509.17641/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 43 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 30 of 34)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Phi-4 Technical Report](https://mlsanity.com/paper/2412.08905/) (2024)
- [A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge](https://mlsanity.com/paper/2206.01718/) (2022)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [From Recognition to Cognition: Visual Commonsense Reasoning](https://mlsanity.com/paper/1811.10830/) (2018)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2410.19168/) (2024)
- [Imagine while Reasoning in Space: Multimodal Visualization-of-Thought](https://mlsanity.com/paper/2501.07542/) (2025)
- [Learning to Answer Questions in Dynamic Audio-Visual Scenarios](https://mlsanity.com/paper/2203.14072/) (2022)
- [CREPE: Can Vision-Language Foundation Models Reason Compositionally?](https://mlsanity.com/paper/2212.07796/) (2022)
- [Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little](https://mlsanity.com/paper/2104.06644/) (2021)
- [Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision](https://mlsanity.com/paper/2010.06775/) (2020)
- [Visual Planning: Let's Think Only with Images](https://mlsanity.com/paper/2505.11409/) (2025)
- [Mind's Eye: Grounded Language Model Reasoning through Simulation](https://mlsanity.com/paper/2210.05359/) (2022)
- [Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models](https://mlsanity.com/paper/2404.03622/) (2024)
- [MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control](https://mlsanity.com/paper/2403.12037/) (2024)
- [Epic-Sounds: A Large-scale Dataset of Actions That Sound](https://mlsanity.com/paper/2302.00646/) (2023)
- [AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?](https://mlsanity.com/paper/2412.02611/) (2024)
- [Things not Written in Text: Exploring Spatial Commonsense from Visual Signals](https://mlsanity.com/paper/2203.08075/) (2022)
- [Visual Commonsense in Pretrained Unimodal and Multimodal Models](https://mlsanity.com/paper/2205.01850/) (2022)
- [Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding](https://mlsanity.com/paper/2303.12513/) (2023)
- [AudioTime: A Temporally-aligned Audio-text Benchmark Dataset](https://mlsanity.com/paper/2407.02857/) (2024)
- [Visually-Augmented Language Modeling](https://mlsanity.com/paper/2205.10178/) (2022)

## Built on by (most cited first, 1 of 1)

- [How Auditory Knowledge in LLM Backbones Shapes Audio Language Models: A Holistic Evaluation](https://mlsanity.com/paper/2603.19195/) (2026)

## Abstract (arXiv)

Even without directly hearing sounds, humans can effortlessly reason about auditory properties, such as pitch, loudness, or sound-source associations, drawing on auditory commonsense. In contrast, language models often lack this capability, limiting their effectiveness in multimodal interactions. As an initial step to address this gap, we present AuditoryBench++, a comprehensive benchmark for evaluating auditory knowledge and reasoning in text-only settings. The benchmark encompasses tasks that range from basic auditory comparisons to contextually grounded reasoning, enabling fine-grained analysis of how models process and integrate auditory concepts. In addition, we introduce AIR-CoT, a novel auditory imagination reasoning method that generates and integrates auditory information during inference through span detection with special tokens and knowledge injection. Extensive experiments with recent LLMs and Multimodal LLMs demonstrate that AIR-CoT generally outperforms both the off-the-shelf models and those augmented with auditory knowledge. The project page is available at https://auditorybenchpp.github.io.