# Pay More Attention To Audio: Mitigating Imbalance of Cross-Modal Attention in Large Audio Language Models

arXiv: https://arxiv.org/abs/2509.18816 · HTML: https://mlsanity.com/paper/2509.18816/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 24 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Builds on (most cited first, 24 of 24)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Aligning Large Multimodal Models with Factually Augmented RLHF](https://mlsanity.com/paper/2309.14525/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023)
- [A Survey on Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2402.00253/) (2024)
- [MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark](https://mlsanity.com/paper/2410.19168/) (2024)
- [Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning](https://mlsanity.com/paper/2403.16999/) (2024)
- [Listen, Think, and Understand](https://mlsanity.com/paper/2305.10790/) (2023)
- [Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs](https://mlsanity.com/paper/2407.21771/) (2024)
- [Sources of Hallucination by Large Language Models on Inference Tasks](https://mlsanity.com/paper/2305.14552/) (2023)
- [GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities](https://mlsanity.com/paper/2406.11768/) (2024)
- [Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities](https://mlsanity.com/paper/2503.03983/) (2025)
- [Hallucination Augmented Contrastive Learning for Multimodal Large Language Model](https://mlsanity.com/paper/2312.06968/) (2023)
- [MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix](https://mlsanity.com/paper/2505.13032/) (2025)
- [Baichuan-Omni-1.5 Technical Report](https://mlsanity.com/paper/2501.15368/) (2025)
- [Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models](https://mlsanity.com/paper/2503.02318/) (2025)
- [Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models](https://mlsanity.com/paper/2505.04921/) (2025)
- [More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models](https://mlsanity.com/paper/2505.21523/) (2025)
- [Audio-CoT: Exploring Chain-of-Thought Reasoning in Large Audio Language Model](https://mlsanity.com/paper/2501.07246/) (2025)
- [Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?](https://mlsanity.com/paper/2505.09439/) (2025)

## Built on by (most cited first, 5 of 5)

- [A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models](https://mlsanity.com/paper/2606.17417/) (2026)
- [Unifying Speech Editing Detection and Content Localization via Prior-Enhanced Audio LLMs](https://mlsanity.com/paper/2601.21463/) (2026)
- [Allocation Before Ranking: Decoupled Token Compression for OmniLLMs](https://mlsanity.com/paper/2608.01665/) (2026)
- [Enhancing Audio Reasoning via Semantic Summary Prediction](https://mlsanity.com/paper/2609.20849/) (2026)
- [SpeakerSleuth: Can Large Audio-Language Models Judge Speaker Consistency across Multi-turn Dialogues?](https://mlsanity.com/paper/2601.04029/) (2026)

## Abstract (arXiv)

Large Audio-Language Models (LALMs) often suffer from audio-textual attention imbalance, prioritizing text over acoustic information, particularly in the multi-modal fusion layers of the Transformer architecture. This bias hinders their ability to fully utilize acoustic cues, causing suboptimal performance on audio reasoning tasks. To mitigate this, we propose MATA, a novel training-free method that dynamically pushes LALMs to pay More Attention To Audio tokens within the self-attention mechanism. Specifically, MATA intervenes post raw attention scoring, targeting only the last token in intermediate layers without introducing additional parameters or computational overhead. Experiments on the MMAU and MMAR benchmarks confirm MATA's effectiveness, with consistent performance gains. Notably, on MMAR, MATA enables an open-source model to surpass the proprietary Gemini 2.0 Flash for the first time. Our work provides an efficient solution to mitigate attention bias and opens a new research direction for enhancing the audio-processing capabilities of multi-modal models.