Bridging What the Model Thinks and How It Speaks: Expressive Speech Generation via Self-Aware Intent-Realization Alignment
On the map: LLM reasoning → Audio-language model reasoning
Abstract (arXiv)
Speech Language Models (SLMs) exhibit strong semantic understanding, yet often fail to translate this capacity into expressive acoustic realization, producing speech with flattened prosody and misaligned emotion. We identify this mismatch as the semantic understanding-acoustic realization gap. Existing approaches typically rely on externally specified proxies, such as emotion labels or style prompts, which require annotations and struggle to capture dynamically evolving expressive intent throughout dialogue. To overcome these limitations, we propose SASLM (Self-Aware Speech Language Model), a proxy-free framework that bridges what the model thinks and how it speaks through self-aware intent-realization alignment: (1) Intent-Aware Bridging self-distills expressive intent from the model's own evolving semantic generation states via a Variational Information Bottleneck (VIB), thereby guiding expressive speech realization without external expressive supervision; while (2) Realization-Aware Alignment reflectively aligns generated acoustics with intended expression through self-reward optimization, progressively improving intent-realization consistency during speech generation. Despite using only 3B parameters and 800 hours of expressive speech data, SASLM achieves state-of-the-art performance on EchoMind among open-source systems, surpassing models over 10 times larger and approaching commercial systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 51 references that are in the map. First posted on arXiv 13 Apr 2026.
Builds on
25 of 51 references.
Closest ancestors · 1
Same direction, newest first
- 2025Qwen2.5-Omni Technical Report26 Mar
Other references · 24
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2024GPT-4o System Cardcited by 4,588
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
- 2025Qwen3-Omni Technical Reportcited by 497
- 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 366
- 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
- 2024GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbotcited by 224
- 2025Kimi-Audio Technical Reportcited by 222
- 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
- 2025Step-Audio: Unified Understanding and Generation in Intelligent Speech Interactioncited by 83
12 more
- 2025Fun-Audio-Chat Technical Reportcited by 19
- 2025LUCY: Linguistic Understanding and Control Yielding Early Stage of Hercited by 17
- 2025Differentiable Reward Optimization for LLM based TTS systemcited by 15
- 2025SpeechJudge: Towards Human-Level Judgment for Speech Naturalnesscited by 15
- 2025Scaling Analysis of Interleaved Speech-Text Language Modelscited by 14
- 2025Self-Improvement for Audio Large Language Model using Unlabeled Speechcited by 10
- 2025VStyle: A Benchmark for Voice Style Adaptation with Spoken Instructionscited by 9
- 2024Style-Talker: Finetuning Audio Language Model and Style-Based Text-to-Speech Model for Fast Spoken Dialogue Generationcited by 7
- 2025EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detectioncited by 3
- 2025Empathy Omni: Enabling Empathetic Speech Response Generation through Large Language Modelscited by 3
- 2025Explore the Reinforcement Learning for the LLM based ASR and TTS systemcited by 3
- 2025UltraVoice: Scaling Fine-Grained Style-Controlled Speech Conversations for Spoken Dialogue Modelscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Speech Captions: Speech-Rewarded Style Planning for Conversational Text-to-Speech
- 2025RLAIF-SPA: Structured AI Feedback for Semantic-Prosodic Alignment in Speech Synthesis
- 2025Optimizing Speech Language Models for Acoustic Consistency
- 2026Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models
- 2026WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training
- 2025Decoding the Ear: A Framework for Objectifying Expressiveness from Human Preference Through Efficient Alignment
- 2026MindFlow: Harmonizing Cognitive Semantics and Acoustic Dynamics for Facial Animation Generation in Dyadic Conversations
- 2026ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models