mlsanityData to 14 Oct 2026

AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Models

arXiv 2506.05140 ↗First posted 5 Jun 2025cs.CL cs.AI cs.SDcited by 18

Not on the map. Why?

Abstract (arXiv)

Understanding the internal mechanisms of large audio-language models (LALMs) is crucial for interpreting their behavior and improving performance. This work presents the first in-depth analysis of how LALMs internally perceive and recognize auditory attributes. By applying vocabulary projection on three state-of-the-art LALMs, we track how attribute information evolves across layers and token positions. We find that attribute information generally decreases with layer depth when recognition fails, and that resolving attributes at earlier layers correlates with better accuracy. Moreover, LALMs heavily rely on querying auditory inputs for predicting attributes instead of aggregating necessary information in hidden states at attribute-mentioning positions. Based on our findings, we demonstrate a method to enhance LALMs. Our results offer insights into auditory attribute processing, paving the way for future improvements.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 67 of its 73 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

67 of 73 references.

References · 67

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2023A Survey of Large Language Modelscited by 2,576
  7. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  9. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  10. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  11. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  12. 2021Knowledge Neurons in Pretrained Transformerscited by 601
38 more
  1. 2024Qwen2-Audio Technical Reportcited by 560
  2. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  3. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  4. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  5. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  6. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  7. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 288
  8. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  9. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
  10. 2023Listen, Think, and Understandcited by 209
  11. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 169
  12. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  13. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 156
  14. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 144
  15. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 128
  16. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  17. 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 122
  18. 2024WavLLM: Towards Robust and Adaptive Speech Large Language Modelcited by 109
  19. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 104
  20. 2020The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modelingcited by 101
  21. 2023Joint Audio and Speech Understandingcited by 100
  22. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 98
  23. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 90
  24. 2022HEAR: Holistic Evaluation of Audio Representationscited by 78
  25. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
  26. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 77
  27. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 71
  28. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  29. 2024Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Taskscited by 71
  30. 2024SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Wordscited by 68
  31. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  32. 2024The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audiocited by 53
  33. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Datacited by 45
  34. 2025SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Informationcited by 42
  35. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  36. 2023MARBLE: Music Audio Representation Benchmark for Universal Evaluationcited by 38
  37. 2023Neuron-Level Knowledge Attribution in Large Language Modelscited by 37
  38. 2024BLSP-Emo: Towards Empathetic Large Speech-Language Modelscited by 36

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 15.

18 papers.

Sort:

By area:

  1. 2026Do Audio-Visual Large Language Models Really See and Hear?cited by 9
  2. 2026All That Glitters Is Not Audio: Rethinking Text Priors and Audio Reliance in Audio-Language Evaluationcited by 8
  3. 2025SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Modelscited by 5
  4. 2026The Cascade Equivalence Hypothesis: When Do Speech LLMs Behave Like ASRrightarrowLLM Pipelines?cited by 5
  5. 2026Causal Tracing of Audio-Text Fusion in Large Audio Language Modelscited by 5
  6. 2025Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variationscited by 4
  7. 2025Do Language Models Associate Sound with Meaning? A Multimodal Study of Sound Symbolismcited by 3
  8. 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Modelscited by 2
  9. 2026Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Modelscited by 2
  10. 2026Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Modelscited by 2
  11. 2026How Auditory Knowledge in LLM Backbones Shapes Audio Language Models: A Holistic Evaluationcited by 2
  12. 2026Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddingscited by 1
6 more
  1. 2026Separating Decision-Rule Misalignment from Readout-Coverage Limitations in Speech Language Modelscited by 1
  2. 2026Tracing Audio Grounding and Answer Selection in Audio LLMscited by 1
  3. 2026Enhancing Audio Reasoning via Semantic Summary Predictioncited by 1
  4. 2026Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Modelscited by 0
  5. 2026Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Modelscited by 0
  6. 2026Can We Read the Mind of an Audio LLM? A Verbalizable, Multilingual Middle-Layer Workspacecited by 0