mlsanityData to 14 Oct 2026

Causal Tracing of Audio-Text Fusion in Large Audio Language Models

arXiv 2603.13768 ↗First posted 14 Mar 2026cs.SD cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Despite the strong performance of large audio language models (LALMs) in various tasks, exactly how and where they integrate acoustic features with textual context remains unclear. We adapt causal tracing to investigate the internal information flow of LALMs during audio comprehension. By conducting layer-wise and token-wise analyses across DeSTA, Qwen, and Voxtral, we evaluate the causal effects of individual hidden states. Layer-wise analysis identifies different fusion strategies, from progressive integration in DeSTA to abrupt late-stage fusion in Qwen. Token-wise analysis shows that the final sequence token acts as an informational bottleneck where the network decisively retrieves relevant information from the audio. We also observe an attention-like query mechanism at intermediate token positions that triggers the model to pull task-relevant audio context. These findings provide a clear characterization of when and where multi-modal integration occurs within LALMs.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 46 of its 53 references that are in the map. First posted on arXiv 14 Mar 2026.

Builds on

46 of 53 references.

References · 46

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  6. 2023Qwen Technical Reportcited by 3,344
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  9. 2024Qwen2 Technical Reportcited by 2,120
  10. 2019Are Sixteen Heads Really Better than One?cited by 816
  11. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  12. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
34 more
  1. 2024Qwen2-Audio Technical Reportcited by 560
  2. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  3. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  4. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 451
  5. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 388
  6. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  7. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  8. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  9. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
  10. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  11. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 144
  12. 2025MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mixcited by 117
  13. 2023Do Language Models Know When They're Hallucinating References?cited by 116
  14. 2026Ministral 3cited by 114
  15. 2024WavLLM: Towards Robust and Adaptive Speech Large Language Modelcited by 109
  16. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
  17. 2023Joint Audio and Speech Understandingcited by 100
  18. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  19. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  20. 2023Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIPcited by 62
  21. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 50
  22. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Datacited by 45
  23. 2025SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Informationcited by 42
  24. 2017Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systemscited by 32
  25. 2024Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoningcited by 31
  26. 2024What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluationcited by 30
  27. 2025Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigationcited by 23
  28. 2024Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Modelscited by 21
  29. 2019Probing the Information Encoded in X-vectorscited by 18
  30. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  31. 2024Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Modelscited by 16
  32. 2024AND: Audio Network Dissection for Interpreting Deep Acoustic Modelscited by 7
  33. 2025BRACE: A Benchmark for Robust Audio Caption Quality Evaluationcited by 7
  34. 2024Property Neurons in Self-Supervised Speech Transformerscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Modelscited by 4
  2. 2026Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddingscited by 1
  3. 2026Tracing Audio Grounding and Answer Selection in Audio LLMscited by 1
  4. 2026On Temporal Binding in Large Audio Language Modelscited by 0
  5. 2026Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucinationcited by 0