mlsanityData to 7 Oct 2026

ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval

arXiv 2605.03361 ↗First posted 5 May 2026cs.AIcited by 0

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Existing audio retrieval benchmarks primarily assess semantic matching, lacking comprehensive evaluation of the logical reasoning capabilities required by complex queries. We introduce ReasonAudio, a benchmark for reasoning-intensive Text-Audio Retrieval that evaluates four abilities: negation, temporal order, sound co-occurrence, and sound duration. It comprises five synthetic subtasks with 1,000 queries over 10,000 composite audio clips and a natural subtask with 100 queries over 1,000 real-world clips. Evaluation of 11 state-of-the-art retrieval systems reveals substantial limitations: the best-performing model, OmniEmbed-7B, achieves an overall score of 20.7. In a controlled experiment designed to reduce the influence of sound-event matching, OmniEmbed-7B attains 53.8% average accuracy, compared with 70.6% for its generative backbone, Qwen2.5-Omni-7B-Thinker, and 95.6% for humans. Our results highlight the challenges of reasoning-intensive audio retrieval and reveal a performance gap between OmniEmbed-7B and its generative backbone.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 38 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

26 of 38 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  3. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  4. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  5. 2019Clotho: An Audio Captioning Datasetcited by 354
  6. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
  7. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 335
  8. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
  9. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 153
  10. 2021The Benefit Of Temporally-Strong Labels In Audio Event Classificationcited by 83
  11. 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83
  12. 2018Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehensioncited by 74
14 more
  1. 2025Vision-Language Models Do Not Understand Negationcited by 69
  2. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  3. 2021Audio Retrieval with Natural Language Queries: A Benchmark Studycited by 61
  4. 2025Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modalitycited by 30
  5. 2025Diffusion vs. Autoregressive Language Models: A Text Embedding Perspectivecited by 23
  6. 2025FreeAudio: Training-Free Timing Planning for Controllable Long-Form Text-to-Audio Generationcited by 21
  7. 2025Scaling Language-Centric Omnimodal Representation Learningcited by 19
  8. 2024MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answeringcited by 17
  9. 2025MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrievalcited by 12
  10. 2026e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddingscited by 12
  11. 2026Massive Sound Embedding Benchmark (MSEB)cited by 9
  12. 2024Dissecting Temporal Understanding in Text-to-Audio Retrievalcited by 3
  13. 2025AudioSet-R: A Refined AudioSet with Multi-Stage LLM Label Reannotationcited by 1
  14. 2026PolyBench: A Benchmark for Compositional Reasoning in Polyphonic Audiocited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026AudioRAG: A Challenging Benchmark for Audio Reasoning and Information Retrieval
  2. 2025Mellow: a small audio language model for reasoning
  3. 2025Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models
  4. 2026A Benchmark for Audio Reasoning Capabilities of Multimodal Large Language Models
  5. 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
  6. 2025Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning
  7. 2024Audiopedia: Audio QA with Knowledge
  8. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities