Auditing generative audio calls for known-task audio-llm evaluation
Not on the map. Why?
Abstract (arXiv)
Speech and audio LLMs are evaluated by comparing waveform predictions with predictions from an automatic speech recognition (ASR) transcript. For fixed closed-set tasks, this conflates acoustic evidence with the need to invoke a generative audio model. We estimate incremental call value with matched selectors sharing pre-call evidence. Each policy may retain the transcript label, use a local encoder, or invoke a generative model; matched control removes generative actions but preserves pre-call evidence and development selection. On VocalSound, transcript-only accuracy is 0.296, while supervised CLAP and WavLM controls reach 0.850 and 0.854 without calls. Full selector reaches 0.925 at 12.5% calls versus 0.921 for matched No-call selector (difference 0.004; 95% CI [-0.025, 0.033]). Thus, results do not show a call gain after transcript and encoder evidence are available. Relevant quantity is incremental accuracy from allowing calls, not the waveform-transcript gap.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 31 references that are in the map. First posted on arXiv 28 Aug 2026.
Builds on
29 of 31 references.
References · 29
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
- 2017Selective Classification for Deep Neural Networkscited by 852
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
- 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
- 2021AST: Audio Spectrogram Transformercited by 643
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
- 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
- 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
17 more
- 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
- 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
- 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
- 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
- 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 173
- 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 128
- 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
- 2025Step-Audio 2 Technical Reportcited by 105
- 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
- 2024Recent Advances in Speech Language Models: A Surveycited by 87
- 2024A Survey on Speech Large Language Models for Understandingcited by 71
- 2022Vocalsound: A Dataset for Improving Human Vocal Sounds Recognitioncited by 62
- 2026Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Musiccited by 19
- 2021An evaluation of word-level confidence estimation for end-to-end automatic speech recognitioncited by 18
- 2024Just ASR + LLM? A Study on Speech Large Language Models' Ability to Identify and Understand Speaker in Spoken Dialoguecited by 16
- 2026HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Modelscited by 8
- 2026The SLT 2026 SmartGlasses Challenge: Benchmarking Egocentric Multi-Talker Speech Recognition and Understanding with Audio-Language Modelscited by 1
Built on by
Not cited yet.