AcoustiClaim: A Numeric Claim Benchmark with Instrument Ground Truth
Not on the map. Why?
Abstract (arXiv)
Audio language models state numbers for acoustic quantities, and neither human opinion nor a judge model says whether such a number is true of the signal. AcoustiClaim extracts each numeric claim from free text, scores it against the instrument that defines the quantity, and classes each quantity by where its reference can be read. Four open-weight systems and one closed model, asked for ten quantities five ways on two corpora, fill 207 cells. Of these, 49 emit fewer than five distinct values, and eight of the 158 cells that can be ranked exceed a rank correlation of 0.3, the bar we set, three with an interval clear of it, five of them one closed model reading pitch. Error sits at or above a constant-predictor floor in every ranked cell but three. The reference decoder we train declines the five voice quantities in prose on 95% of mixtures, with nothing withheld, and states them on the clean twins, reproducing its targets' rule from audio alone. With a calibrated threshold, withholding lowers error on all ten quantities on the mixtures in the mean and on eight at every split, against at most 0.6% from a random selector. A linear baseline orders errors at least as well as ours. F0 s.d. and shimmer stay above the constant floor.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 24 Sep 2026.
Builds on
19 of 25 references.
References · 19
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2017Selective Classification for Deep Neural Networkscited by 852
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2024Qwen2-Audio Technical Reportcited by 560
- 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
- 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
- 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 219
- 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 192
- 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
7 more
- 2023TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudiocited by 59
- 2025Granite-speech: open-source speech-aware LLMs with strong English ASR capabilitiescited by 27
- 2025Audio Large Language Models Can Be Descriptive Speech Quality Evaluatorscited by 25
- 2024Salmon: A Suite for Acoustic Language Model Evaluationcited by 24
- 2025QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptionscited by 13
- 2025CoLMbo: Speaker Language Model for Descriptive Profilingcited by 5
- 2025The MUSE Benchmark: Probing Music Perception and Auditory Relational Reasoning in Audio LLMScited by 4
Built on by
Not cited yet.