Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models
Not on the map. Why?
Abstract (arXiv)
Emotion is a central dimension of spoken communication, yet, we still lack a mechanistic account of how modern large audio-language models (LALMs) encode it internally. We present the first neuron-level interpretability study of emotion-sensitive neurons (ESNs) in LALMs and provide causal evidence supporting the existence of such units in Qwen2.5-Omni, Kimi-Audio, and Audio Flamingo 3. Across these three widely used open-source models, we compare frequency-, entropy-, mean-deviation-, and contrast-based neuron selectors on multiple emotion recognition benchmarks. Using inference-time interventions, we reveal a consistent emotion-specific signature: deactivating neurons selected for a given emotion disproportionately degrades recognition of that emotion while largely preserving other classes, whereas targeted steering amplifies these units to bias predictions toward the target emotion. These effects arise with modest amounts of identification data and scale systematically with intervention strength. We further observe that ESNs exhibit non-uniform layer-wise clustering with partial cross-dataset transfer. Taken together, our results offer a causal, neuron-level account of emotion decisions in LALMs and highlight targeted neuron interventions as an actionable handle for controllable affective behaviors.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 22 of its 47 references that are in the map. First posted on arXiv 6 Jan 2026.
Builds on
22 of 47 references.
References · 22
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2020GLU Variants Improve Transformercited by 1,724
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
- 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
- 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
- 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 196
- 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
10 more
- 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
- 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
- 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 110
- 2024Universal Neurons in GPT2 Language Modelscited by 100
- 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
- 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 61
- 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Unitscited by 34
- 2020Expressive TTS Training with Frame and Style Reconstruction Losscited by 30
- 2024Measuring the Inconsistency of Large Language Models in Preferential Rankingcited by 18
- 2025Do LLMs "Feel"? Emotion Circuits Discovery and Controlcited by 13
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Modelscited by 2
- 2026Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMscited by 1
- 2026AIPsy-Affect: A Keyword-Free Clinical Stimulus Battery for Mechanistic Interpretability of Emotion in Language Modelscited by 0
- 2026Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognitioncited by 0
- 2026Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Modelscited by 0
- 2026Tracing Audio Grounding and Answer Selection in Audio LLMscited by 0
- 2026Tracing Audio Grounding and Answer Selection in Audio LLMs4 Sep 2026
- 2026Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Models17 Aug 2026
- 2026Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognition5 Jun 2026
- 2026AIPsy-Affect: A Keyword-Free Clinical Stimulus Battery for Mechanistic Interpretability of Emotion in Language Models26 Apr 2026
- 2026Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models18 Mar 2026
- 2026Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs15 Mar 2026