# SCENEBench: An Audio Understanding Benchmark Grounded in Assistive and Industrial Use Cases

arXiv: https://arxiv.org/abs/2603.09853 · HTML: https://mlsanity.com/paper/2603.09853/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 14 of 14)

- [MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2410.19168/) (2024)
- [AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension](https://mlsanity.com/paper/2402.07729/) (2024)
- [Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models](https://mlsanity.com/paper/2507.08128/) (2025)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [Clotho-AQA: A Crowdsourced Dataset for Audio Question Answering](https://mlsanity.com/paper/2204.09634/) (2022)
- [Semantic Audio-Visual Navigation](https://mlsanity.com/paper/2012.11583/) (2020)
- [DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speech](https://mlsanity.com/paper/2207.01063/) (2022)
- [MIMII Dataset: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection](https://mlsanity.com/paper/1909.09347/) (2019)
- [Single-Channel Multi-Speaker Separation using Deep Clustering](https://mlsanity.com/paper/1607.02173/) (2016)
- [DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data](https://mlsanity.com/paper/2409.20007/) (2024)
- [ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detection](https://mlsanity.com/paper/1908.03299/) (2019)
- [ImageNot: A contrast with ImageNet preserves model rankings](https://mlsanity.com/paper/2404.02112/) (2024)
- [DCASE 2018 Challenge - Task 5: Monitoring of domestic activities based on multi-channel acoustics](https://mlsanity.com/paper/1807.11246/) (2018)

## Built on by (most cited first, 3 of 3)

- [MuLA-Bench: A Multilingual Long-Form Audio Understanding Benchmark via Multi-Tier Auditing](https://mlsanity.com/paper/2609.23416/) (2026)
- [EXAM2: Extending Audio Understanding in Multilingual and Multimodal Analysis](https://mlsanity.com/paper/2608.23758/) (2026)
- [Long-Term Memory-Guided Enhancement for Target Perception in Audio-Language Models](https://mlsanity.com/paper/2609.36577/) (2026)

## Abstract (arXiv)

Advances in large language models (LLMs) have enabled significant capabilities in audio processing, resulting in state-of-the-art models now known as Large Audio Language Models (LALMs). However, minimal work has been done to measure audio understanding beyond automatic speech recognition (ASR). This paper closes that gap by proposing a benchmark suite, SCENEBench (Spatial, Cross-lingual, Environmental, Non-speech Evaluation), that targets a broad form of audio comprehension across four real-world categories: background sound understanding, noise localization, cross-linguistic speech understanding, and vocal characterizer recognition. These four categories are selected based on understudied needs from accessibility technology and industrial noise monitoring. In addition to performance, we also measure model latency. The purpose of this benchmark suite is to assess audio beyond just what words are said - rather, how they are said and the non-speech components of the audio. Because our audio samples are synthetically constructed (e.g., by overlaying two natural audio samples), we further validate our benchmark against 20 natural audio items per task, sub-sampled from existing datasets to match our task criteria, to assess ecological validity. We assess five state-of-the-art LALMs and find critical gaps: performance varies across tasks, with some tasks performing below random chance and others achieving high accuracy. These results provide direction for targeted improvements in model capabilities.