# SCIC: Scope- and Codebook-Aware Instruction Conditioning for Speaker-Adapted Expressive TTS

arXiv: https://arxiv.org/abs/2609.39088 · HTML: https://mlsanity.com/paper/2609.39088/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 25 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 23 of 23)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization](https://mlsanity.com/paper/2601.05242/) (2026)
- [CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training](https://mlsanity.com/paper/2505.17589/) (2025)
- [Qwen3-TTS Technical Report](https://mlsanity.com/paper/2601.15621/) (2026)
- [Qwen3-ASR Technical Report](https://mlsanity.com/paper/2601.21337/) (2026)
- [Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens](https://mlsanity.com/paper/2503.01710/) (2025)
- [PromptTTS: Controllable Text-to-Speech with Text Descriptions](https://mlsanity.com/paper/2211.12171/) (2022)
- [Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit](https://mlsanity.com/paper/2210.17016/) (2022)
- [InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Prompt](https://mlsanity.com/paper/2301.13662/) (2023)
- [PromptTTS 2: Describing and Generating Voices with Text Prompt](https://mlsanity.com/paper/2309.02285/) (2023)
- [PromptStyle: Controllable Style Transfer for Text-to-Speech with Natural Language Descriptions](https://mlsanity.com/paper/2305.19522/) (2023)
- [Fish Audio S2 Technical Report](https://mlsanity.com/paper/2603.08823/) (2026)
- [OV-InstructTTS: Towards Open-Vocabulary Instruct Text-to-Speech](https://mlsanity.com/paper/2601.01459/) (2026)
- [FlexiVoice: Enabling Flexible Style Control in Zero-Shot TTS with Natural Language Instructions](https://mlsanity.com/paper/2601.04656/) (2026)
- [FleSpeech: Flexibly Controllable Speech Generation with Various Prompts](https://mlsanity.com/paper/2501.04644/) (2025)
- [ReStyle-TTS: Relative and Continuous Style Control for Zero-Shot Speech Synthesis](https://mlsanity.com/paper/2601.03632/) (2026)
- [Qwen-Audio-3.0-TTS: Freely Controllable and Highly Robust Speech Synthesis with Multi-Stage Training Paradigm](https://mlsanity.com/paper/2607.23938/) (2026)
- [HD-PPT: Hierarchical Decoding of Content- and Prompt-Preference Tokens for Instruction-based TTS](https://mlsanity.com/paper/2509.19001/) (2025)
- [VoXtream2: Full-stream TTS with dynamic speaking rate control](https://mlsanity.com/paper/2603.13518/) (2026)
- [MAGIC-TTS: Fine-Grained Controllable Speech Synthesis with Explicit Local Duration and Pause Control](https://mlsanity.com/paper/2604.21164/) (2026)
- [EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis](https://mlsanity.com/paper/2608.23791/) (2026)
- [Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis](https://mlsanity.com/paper/2606.22811/) (2026)
- [WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS](https://mlsanity.com/paper/2607.06461/) (2026)

## Abstract (arXiv)

Long-form live-streaming TTS requires context-dependent prosody and paragraph-level coherence. However, many existing instruction-based TTS systems use global or uniform conditions, providing limited explicit control over clause-level relative prosodic changes. We introduce Speaker-Relative Inline Prosody Control, where each Pitch, Energy, or Speed instruction targets a clause relative to the preceding clause from the same speaker, while Pause uses an absolute duration interval. In codec-based TTS, Speed and Pause affect sequence length, whereas Pitch and Energy rely on residual codebooks. By analyzing Qwen3-TTS RVQ codebooks, we find that Energy concentrates in early residual codebooks, whereas Pitch accumulates across a deeper prefix. We therefore propose Scope- and Codebook-Aware Instruction Conditioning (SCIC), combining a Temporal Instruction Router for frame-level tag activation with Tag-Specific Codebook Weighting over residual codebooks. SCIC improves speaker-relative Pitch and Energy control over standard instruction fine-tuning using text-token tags. We further apply multi-reward GDPO post-training to jointly optimize control and quality, improving control accuracy while preserving CER and speaker similarity. In long-form synthesis, SCIC produces a more distinct paragraph-level expressive hierarchy than speaker-adapted SFT without instructions. Audio demos are available at: https://taoliveaigc.github.io/SCIC/