mlsanityData to 14 Oct 2026

Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding

arXiv 2506.07233 ↗First posted 8 Jun 2025eess.AS cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Large Audio-Language Models (LALMs) can take audio and text as the inputs and answer questions about the audio. While prior LALMs have shown strong performance on standard benchmarks, there has been alarming evidence that LALMs can hallucinate what is presented in the audio. To mitigate the hallucination of LALMs, we introduce Audio-Aware Decoding (AAD), a lightweight inference-time strategy that uses contrastive decoding to compare the token prediction logits with and without the audio context. By contrastive decoding, AAD promotes the tokens whose probability increases when the audio is present. We conduct our experiment on object hallucination datasets with three LALMs and show that AAD improves the F1 score by 0.046 to 0.428. We also show that AAD can improve the accuracy on general audio QA datasets like Clotho-AQA by 5.4% to 10.3%. We conduct thorough ablation studies to understand the effectiveness of each component in AAD.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 36 of its 39 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

36 of 39 references.

References · 36

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  6. 2024Qwen2 Technical Reportcited by 2,120
  7. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  8. 2023A Survey on Multimodal Large Language Modelscited by 909
  9. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  10. 2023Instruction Tuning with GPT-4cited by 686
  11. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  12. 2024Qwen2-Audio Technical Reportcited by 560
24 more
  1. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  2. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  3. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  4. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 327
  5. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  6. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  7. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  8. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 137
  9. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 133
  10. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 128
  11. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  12. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
  13. 2023Joint Audio and Speech Understandingcited by 100
  14. 2022Clotho-AQA: A Crowdsourced Dataset for Audio Question Answeringcited by 85
  15. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  16. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Datacited by 45
  17. 2025SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Informationcited by 42
  18. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  19. 2024DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignmentcited by 33
  20. 2024Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Modelscited by 31
  21. 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 31
  22. 2025Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Modelscited by 27
  23. 2024On the Audio Hallucinations in Large Audio-Video Language Modelscited by 15
  24. 2025Delta -- Contrastive Decoding Mitigates Text Hallucinations in Large Language Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 22.

24 papers.

Sort:

By area:

  1. 2025Silence is Golden: Mitigating Hallucinations in Large Audio-Language Models via Layer-Weighted Vector Steeringcited by 6
  2. 2026How Contrastive Decoding Enhances Large Audio Language Modelscited by 6
  3. 2026ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Modelscited by 3
  4. 2026Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Modelscited by 3
  5. 2026Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Modelscited by 2
  6. 2026CAAD: Contrastive Audio-Aware Distillation for Efficient Speech Language Modelscited by 2
  7. 2026Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Timecited by 1
  8. 2026Adaptive Perturbation Selection for Contrastive Audio Decodingcited by 1
  9. 2026TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Modelscited by 1
  10. 2025Evaluating Hallucinations in Audio-Visual Multimodal LLMs with Spoken Queries under Diverse Acoustic Conditionscited by 0
  11. 2026CAF-Score: Calibrating CLAP with LALMs for Reference-free Audio Captioning Evaluationcited by 0
  12. 2026Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generationcited by 0
12 more
  1. 2026Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Modelscited by 0
  2. 2026Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Modelscited by 0
  3. 2026Attention-Guided Reliability Scaling for Contrastive Decoding in Robust Audio-Visual Speech Recognitioncited by 0
  4. 2026Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactionscited by 0
  5. 2026Tri-PvP: Exposing Modality Bias in Omni-Modal Large Language Models through Perceptual-Propositional Evidence Conflictscited by 0
  6. 2026Music Hallucination in Audio-Language Models: A Hierarchical Formulation and Empirical Studycited by 0
  7. 2026REVE: Efficient Hallucination Correction for Large Audio-Language Models via Reused Encoder Statescited by 0
  8. 2026Reward-Tilted On-Policy Distillation for Acoustic Grounding in Audio-Language Modelscited by 0
  9. 2026Broadening Uncertainty Estimation for Audio Question Answering Across Methods, Formats, and Inputscited by 0
  10. 2026MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Modelscited by 0
  11. 2026AnchorPrompt: Self-Distilled Soft Prompts for Robust Audio-Language Modelscited by 0
  12. 2026Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucinationcited by 0