mlsanityData to 14 Oct 2026

Are Audio-Language Models Listening? Audio-Specialist Heads for Adaptive Audio Steering

arXiv 2603.06854 ↗First posted 6 Mar 2026cs.SD cs.AIcited by 4

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Multimodal large language models can exhibit text dominance, over-relying on linguistic priors instead of grounding predictions in non-text inputs. One example is large audio-language models (LALMs) where decisive audio evidence can be under-utilized even when it contains important information. To address this issue we use mechanistic interpretability to identify a small set of audio-specialist attention heads whose audio attention yields a ``listening'' signal. We show that this signal increases when audio evidence affects the model's output, providing an indicator of audio engagement under standard prompting. Leveraging this localization, we construct an audio--silence steering direction and apply an inference-time activation intervention to the final representation, amplifying the model's audio effect. To demonstrate the utility of this intervention, we show on MMAU that this improves accuracy by up to +8.0 percentage points on two Qwen-based LALMs, without any parameter updates.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 31 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

29 of 31 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  6. 2022In-context Learning and Induction Headscited by 964
  7. 2023Steering Language Models With Activation Engineeringcited by 922
  8. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  9. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  11. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  12. 2024Qwen2-Audio Technical Reportcited by 560
17 more
  1. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  2. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  3. 2024Multi-Modal Hallucination Control by Visual Information Groundingcited by 208
  4. 2025Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilitiescited by 133
  5. 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answeringcited by 65
  6. 2025MLLMs are Deeply Affected by Modality Biascited by 40
  7. 2024Debiasing Multimodal Large Language Models via Penalization of Language Priorscited by 37
  8. 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 31
  9. 2024What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluationcited by 30
  10. 2025Do Audio LLMs Really LISTEN, or Just Transcribe? Measuring Lexical vs. Acoustic Emotion Cues Reliancecited by 24
  11. 2025Head Pursuit: Probing Attention Specialization in Multimodal Transformerscited by 17
  12. 2025When Audio and Text Disagree: Revealing Text Bias in Large Audio-Language Modelscited by 13
  13. 2025Beyond Transcription: Mechanistic Interpretability in ASRcited by 13
  14. 2025Silence is Golden: Mitigating Hallucinations in Large Audio-Language Models via Layer-Weighted Vector Steeringcited by 6
  15. 2026When Audio-LLMs Don't Listen: A Cross-Linguistic Study of Modality Arbitrationcited by 5
  16. 2024Finding Task-specific Subnetworks in Multi-task Spoken Language Understanding Modelcited by 2
  17. 2025An uncertainty-aware framework for data-efficient multi-view animal pose estimationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Tracing Audio Grounding and Answer Selection in Audio LLMscited by 1
  2. 2026Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Modelscited by 0
  3. 2026Inherited Heads: Audio language models track speakers with their text backbone's attention, and an attention-mass ranking retrieves a different setcited by 0
  4. 2026On Temporal Binding in Large Audio Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Steering Where to Listen: Instruction-Based Activation Steering Redirects Temporal Attention in Large Audio-Language Models
  2. 2026A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models
  3. 2026Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models
  4. 2026Continuous Audio Thinking for Large Audio Language Models
  5. 2025Thinking with Sound: Audio Chain-of-Thought Enables Multimodal Reasoning in Large Audio-Language Models
  6. 2026Selective Listening: Mechanism-Guided Control of Audio Influence in Large Audio-Language Models
  7. 2026Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoning
  8. 2025Audio-Maestro: Enhancing Large Audio-Language Models with Tool-Augmented Reasoning