mlsanityData to 14 Oct 2026

Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech

arXiv 2606.01479 ↗First posted 31 May 2026cs.CLcited by 3

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Integrating large language models (LLMs) into text-to-speech (TTS) systems has improved speech expressiveness, yet interpretable emotional control remains challenging. Existing approaches primarily rely on external conditioning or global activation steering, offering limited insight into the internal representations underlying emotional control. In this work, we analyze emotion-related variation in the semantic hidden states of LLM-based TTS models using sparse autoencoders (SAEs) to identify sparse latent features. Our analysis shows that emotional variation is distributed across multiple sparse latent features, while intervening on a small subset enables interpretable emotion control. Building on this observation, we introduce a feature-level intervention framework for bidirectional emotion induction and suppression without modifying backbone parameters. We further show that distinct latent features are associated with specific acoustic attributes (e.g., pitch), suggesting that emotional expression arises from coordinated latent contributions rather than a single global shift. Empirically, steering these sparse latent features achieves comparable or superior emotion induction and suppression performance relative to global steering and existing TTS baselines.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 39 of its 47 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

39 of 47 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Learning Interpretable Features in Audio Latent Spaces via Sparse Autoencoders27 Oct
  2. 2025EmoSteer-TTS: Fine-Grained and Training-Free Emotion-Controllable Text-to-Speech via Activation Steering5 Aug
  3. 2022Extracting Latent Steering Vectors from Pretrained Language Models10 May

Other references · 36

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  3. 2022Toy Models of Superpositioncited by 971
  4. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  5. 2024Scaling and evaluating sparse autoencoderscited by 553
  6. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  7. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  8. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  9. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  10. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  11. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 245
  12. 2013k-Sparse Autoencoderscited by 215
24 more
  1. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 205
  2. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 173
  3. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 167
  4. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  5. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 115
  6. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  7. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 108
  8. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 90
  9. 2025SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoderscited by 80
  10. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 77
  11. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 70
  12. 2024Get What You Want, Not What You Don't: Image Content Suppression for Text-to-Image Diffusion Modelscited by 63
  13. 2023TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Modelscited by 48
  14. 2023An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verificationcited by 42
  15. 2025Steering CLIP's vision transformer with sparse autoencoderscited by 38
  16. 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 34
  17. 2017Emotional End-to-End Neural Speech Synthesizercited by 30
  18. 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 28
  19. 2024Can We Talk Models Into Seeing the World Differently?cited by 23
  20. 2025SAE-V: Interpreting Multimodal Models for Enhanced Alignmentcited by 23
  21. 2021Expressive Text-to-Speech using Style Tagcited by 20
  22. 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Controlcited by 17
  23. 2024Text-aware and Context-aware Expressive Audiobook Speech Synthesiscited by 3
  24. 2025UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speechcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Tracing a Sparse Emotion-Control Circuit in LLM-Based Text-to-Speech
  2. 2026Why Are Some Emotions Harder for LLMs? Uncovering the Causal Mechanisms of Emotion Inference via Sparse Autoencoders
  3. 2026CoCoEmo: Composable and Controllable Human-Like Emotional TTS via Activation Steering
  4. 2024Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Models
  5. 2026Steerspeech: Activation Steering For Emotion Control In Generated Speech
  6. 2026Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoders
  7. 2025Emotion-Aligned Generation in Diffusion Text to Speech Models via Preference-Guided Optimization
  8. 2026EmoShift: Lightweight Activation Steering for Enhanced Emotion-Aware Speech Synthesis