Music Hallucination in Audio-Language Models: A Hierarchical Formulation and Empirical Study
Not on the map. Why?
Abstract (arXiv)
Audio-language models increasingly generate confident music descriptions that are unsupported by the input audio. We present, to our knowledge, the first music-specific, layer-wise, multi-paradigm empirical study of hallucination in audio-language models and formulate it as a hierarchical perceptual grounding failure across five layers: sound events, temporal properties, tonal attributes, style, and emotion. We introduce MuseDiag, a multi-paradigm diagnostic framework with contradiction-based verification, and evaluate nine models (four open-source and five closed-source). We find that (1) vocal misperception is a universal weakness across all nine models, tonal perception is a major axis of architectural differentiation, and Audio-Flamingo-3 remains the stable leader while substantial reordering below it reveals paradigm-specific vulnerability profiles; (2) affirmative bias, generation-mode effects, and layer-specific perceptual limitations are each empirically associated with the observed patterns, with convergent evidence from multiple analyses rather than strict causal attribution; and (3) our two training-free mitigation methods, Audio-Dependency-Aware Decoding for Music (ADD-M) and Taxonomy-Guided Perceptual Anchoring (TPA), can reduce hallucination in probing, but their gains vary by model and often do not carry over to free-form generation, showing that music hallucination mitigation must be evaluated across paradigms.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 57 references that are in the map. First posted on arXiv 24 Jul 2026.
Builds on
25 of 57 references.
References · 25
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
- 2018Object Hallucination in Image Captioningcited by 661
- 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
- 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
- 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluationcited by 293
- 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
- 2023Pengi: An Audio Language Model for Audio Taskscited by 217
- 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
- 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 183
13 more
- 2019pyannote.audio: neural building blocks for speaker diarizationcited by 175
- 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 151
- 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 98
- 2025HalluLens: LLM Hallucination Benchmarkcited by 88
- 2023The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluationcited by 68
- 2023FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Modelscited by 44
- 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Modelscited by 43
- 2023MARBLE: Music Audio Representation Benchmark for Universal Evaluationcited by 38
- 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 29
- 2025AVCD: Mitigating Hallucinations in Audio-Visual Large Language Models through Contrastive Decodingcited by 26
- 2025Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decodingcited by 22
- 2025CMI-Bench: A Comprehensive Benchmark for Evaluating Music Instruction Followingcited by 15
- 2018Revisiting Singing Voice Detection: a Quantitative Review and the Future Outlookcited by 1
Built on by
Not cited yet.