# ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval

arXiv: https://arxiv.org/abs/2605.03361 · HTML: https://mlsanity.com/paper/2605.03361/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 5 May 2026.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Clotho: An Audio Captioning Dataset](https://mlsanity.com/paper/1910.09387/) (2019)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [Wav2CLIP: Learning Robust Audio Representations From CLIP](https://mlsanity.com/paper/2110.11499/) (2021)
- [BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval](https://mlsanity.com/paper/2407.12883/) (2024)
- [The Benefit Of Temporally-Strong Labels In Audio Event Classification](https://mlsanity.com/paper/2105.07031/) (2021)
- [Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction](https://mlsanity.com/paper/2502.11946/) (2025)
- [Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension](https://mlsanity.com/paper/1804.00320/) (2018)
- [Vision-Language Models Do Not Understand Negation](https://mlsanity.com/paper/2501.09425/) (2025)
- [FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions](https://mlsanity.com/paper/2403.15246/) (2024)
- [Audio Retrieval with Natural Language Queries: A Benchmark Study](https://mlsanity.com/paper/2112.09418/) (2021)
- [Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modality](https://mlsanity.com/paper/2505.02466/) (2025)
- [Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective](https://mlsanity.com/paper/2505.15045/) (2025)
- [Scaling Language-Centric Omnimodal Representation Learning](https://mlsanity.com/paper/2510.11693/) (2025)
- [FreeAudio: Training-Free Timing Planning for Controllable Long-Form Text-to-Audio Generation](https://mlsanity.com/paper/2507.08557/) (2025)
- [MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering](https://mlsanity.com/paper/2412.15540/) (2024)
- [e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings](https://mlsanity.com/paper/2601.03666/) (2026)
- [MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval](https://mlsanity.com/paper/2510.09510/) (2025)
- [Massive Sound Embedding Benchmark (MSEB)](https://mlsanity.com/paper/2602.07143/) (2026)
- [Dissecting Temporal Understanding in Text-to-Audio Retrieval](https://mlsanity.com/paper/2409.00851/) (2024)
- [AudioSet-R: A Refined AudioSet with Multi-Stage LLM Label Reannotation](https://mlsanity.com/paper/2508.15429/) (2025)
- [PolyBench: A Benchmark for Compositional Reasoning in Polyphonic Audio](https://mlsanity.com/paper/2603.05128/) (2026)

## Built on by (most cited first, 1 of 1)

- [World Embedding Benchmark](https://mlsanity.com/paper/2610.03632/) (2026)

## Abstract (arXiv)

Existing audio retrieval benchmarks primarily assess semantic matching, lacking comprehensive evaluation of the logical reasoning capabilities required by complex queries. We introduce ReasonAudio, a benchmark for reasoning-intensive Text-Audio Retrieval that evaluates four abilities: negation, temporal order, sound co-occurrence, and sound duration. It comprises five synthetic subtasks with 1,000 queries over 10,000 composite audio clips and a natural subtask with 100 queries over 1,000 real-world clips. Evaluation of 11 state-of-the-art retrieval systems reveals substantial limitations: the best-performing model, OmniEmbed-7B, achieves an overall score of 20.7. In a controlled experiment designed to reduce the influence of sound-event matching, OmniEmbed-7B attains 53.8% average accuracy, compared with 70.6% for its generative backbone, Qwen2.5-Omni-7B-Thinker, and 95.6% for humans. Our results highlight the challenges of reasoning-intensive audio retrieval and reveal a performance gap between OmniEmbed-7B and its generative backbone.