mlsanityData to 14 Oct 2026

Neuron-Level Emotion Control in Speech-Generative Large Audio-Language Models

arXiv 2603.17231 ↗First posted 18 Mar 2026cs.CL eess.AScited by 2

Not on the map. Why?

Abstract (arXiv)

Large audio-language models (LALMs) can produce expressive speech, yet reliable emotion control remains elusive: conversions often miss the target affect and may degrade linguistic fidelity through refusals, hallucinations, or paraphrase. We present, to our knowledge, the first neuron-level study of emotion control in speech-generative LALMs and demonstrate that compact emotion-sensitive neurons (ESNs) are causally actionable, enabling training-free emotion steering at inference time. ESNs are identified via success-filtered activation aggregation enforcing both emotion realization and content preservation. Across three LALMs (Qwen2.5-Omni-7B, MiniCPM-o 4.5, Kimi-Audio), ESN interventions yield emotion-specific gains that generalize to unseen speakers and are supported by automatic and human evaluation. Controllability depends on selector design, mask sparsity, filtering, and intervention strength. Our results establish a mechanistic framework for training-free emotion control in speech generation.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 46 of its 58 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

46 of 58 references.

References · 46

  1. 2020GLU Variants Improve Transformercited by 1,804
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  5. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  6. 2023Steering Language Models With Activation Engineeringcited by 922
  7. 2025Qwen2.5-Omni Technical Reportcited by 736
  8. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  9. 2025Qwen3-Omni Technical Reportcited by 497
  10. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  11. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  12. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 255
34 more
  1. 2025Kimi-Audio Technical Reportcited by 222
  2. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 205
  3. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 173
  4. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 128
  5. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  6. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  7. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 121
  8. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
  9. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  10. 2024Universal Neurons in GPT2 Language Modelscited by 102
  11. 2023Neurons in Large Language Models: Dead, N-gram, Positionalcited by 88
  12. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 68
  13. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 39
  14. 2023Neuron-Level Knowledge Attribution in Large Language Modelscited by 37
  15. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 34
  16. 2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 33
  17. 2020Expressive TTS Training with Frame and Style Reconstruction Losscited by 30
  18. 2020Fine-grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesiscited by 29
  19. 2021Textless Speech Emotion Conversion using Discrete and Decomposed Representationscited by 29
  20. 2022MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesiscited by 29
  21. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Modelcited by 26
  22. 2024EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speechcited by 24
  23. 2025EmoSteer-TTS: Fine-Grained and Training-Free Emotion-Controllable Text-to-Speech via Activation Steeringcited by 23
  24. 2019End-to-End Emotional Speech Synthesis Using Style Tokens and Semi-Supervised Trainingcited by 19
  25. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  26. 2025Do LLMs "Feel"? Emotion Circuits Discovery and Controlcited by 17
  27. 2022iEmoTTS: Toward Robust Cross-Speaker Emotion Transfer and Control for Speech Synthesis based on Disentanglement between Prosody and Timbrecited by 15
  28. 2021An Improved StarGAN for Emotional Voice Conversion: Enhancing Voice Quality and Data Augmentationcited by 7
  29. 2023EMOCONV-DIFF: Diffusion-based Speech Emotion Conversion for Non-parallel and In-the-wild Datacited by 7
  30. 2024AND: Audio Network Dissection for Interpreting Deep Acoustic Modelscited by 7
  31. 2022An Overview & Analysis of Sequence-to-Sequence Emotional Voice Conversioncited by 6
  32. 2026Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Modelscited by 6
  33. 2025Isolating Culture Neurons in Multilingual Large Language Modelscited by 4
  34. 2025Finding Culture-Sensitive Neurons in Vision-Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Modelscited by 0
  2. 2026Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Modelscited by 0