# How Contrastive Decoding Enhances Large Audio Language Models

arXiv: https://arxiv.org/abs/2603.09232 · HTML: https://mlsanity.com/paper/2603.09232/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 30 of 33)

- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2311.16922/) (2023)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023)
- [DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models](https://mlsanity.com/paper/2309.03883/) (2023)
- [MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2410.19168/) (2024)
- [Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models](https://mlsanity.com/paper/2507.08128/) (2025)
- [Listen, Think, and Understand](https://mlsanity.com/paper/2305.10790/) (2023)
- [Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding](https://mlsanity.com/paper/2403.18715/) (2024)
- [Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities](https://mlsanity.com/paper/2402.01831/) (2024)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities](https://mlsanity.com/paper/2406.11768/) (2024)
- [Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities](https://mlsanity.com/paper/2503.03983/) (2025)
- [IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding](https://mlsanity.com/paper/2402.18476/) (2024)
- [MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix](https://mlsanity.com/paper/2505.13032/) (2025)
- [WavLLM: Towards Robust and Adaptive Speech Large Language Model](https://mlsanity.com/paper/2404.00656/) (2024)
- [Joint Audio and Speech Understanding](https://mlsanity.com/paper/2309.14405/) (2023)
- [A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Models](https://mlsanity.com/paper/2405.09589/) (2024)
- [Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks](https://mlsanity.com/paper/2411.05361/) (2024)
- [DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment](https://mlsanity.com/paper/2507.02768/) (2025)
- [Recent Advances in Discrete Speech Tokens: A Review](https://mlsanity.com/paper/2502.06490/) (2025)
- [SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Information](https://mlsanity.com/paper/2505.13237/) (2025)
- [Distilling an End-to-End Voice Assistant Without Instruction Training Data](https://mlsanity.com/paper/2410.02678/) (2024)
- [Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding](https://mlsanity.com/paper/2506.07233/) (2025)
- [Building a Taiwanese Mandarin Spoken Language Model: A First Attempt](https://mlsanity.com/paper/2411.07111/) (2024)
- [Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samples](https://mlsanity.com/paper/2505.14518/) (2025)

## Built on by (most cited first, 6 of 6)

- [Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models](https://mlsanity.com/paper/2604.25591/) (2026)
- [Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models](https://mlsanity.com/paper/2606.05161/) (2026)
- [CAAD: Contrastive Audio-Aware Distillation for Efficient Speech Language Models](https://mlsanity.com/paper/2606.23052/) (2026)
- [Adaptive Perturbation Selection for Contrastive Audio Decoding](https://mlsanity.com/paper/2607.00247/) (2026)
- [Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models](https://mlsanity.com/paper/2607.11801/) (2026)
- [MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models](https://mlsanity.com/paper/2609.33893/) (2026)

## Abstract (arXiv)

While Contrastive Decoding (CD) has been proposed to enhance Large Audio Language Models (LALMs), it has not been evaluated at scale, and the underlying mechanisms driving its success remain unclear. This study systematically evaluates four distinct CD strategies across diverse LALM architectures. We identify Audio-Aware Decoding and Audio Contrastive Decoding as the most effective methods. However, their impact varies significantly across models. To explain this variability, we profile the baseline error composition of each model and measure how readily contrastive decoding corrects each error type. Our analysis demonstrates that CD reliably rectifies errors in which models falsely claim an absence of audio or resort to uncertainty-driven guessing, but is relatively poor at correcting flawed reasoning or confident misassertions. Crucially, CD's benefit closely tracks the composition of a model's baseline error profile: when errors caused by audio ignorance or uncertainty-driven guessing constitute only a small fraction of a model's errors, gains are marginal or even negative. A token-level analysis reveals the underlying mechanism: when the amateur's output is dominated by hesitation markers, CD's suppression naturally targets uncertainty-driven errors while having limited effect on confident misassertions.