mlsanityData to 14 Oct 2026

EmphTTS: an emphasis-control TTS with reinforcement learning

arXiv 2609.27599 ↗First posted 23 Sep 2026eess.AS cs.CLcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Generating controllable and human-like emphasis remains an open challenge in text-to-speech, even when explicit emphasis control signals are provided in the text input, limiting the communicative accuracy of synthetic speech in real-world applications. Reinforcement learning has recently shown promise for post-training TTS systems to align with human preference, yet existing methods have not been applied to word-level prosodic control. We present EmphTTS, a non-autoregressive TTS system that applies Group Relative Policy Optimization (GRPO) to the duration predictor with an emphasis localization reward, enabling direct optimization for word-level emphasis. Evaluations show that EmphTTS achieves the best emphasis controllability and performs the best in emphasis objective evaluation. In subjective preference tests, EmphTTS is significantly preferred over synthetic groundtruth and most baselines. Ablation studies show that GRPO improves emphasis realization beyond supervised-finetuning-based duration modeling and simple speaking-rate adjustment, while alleviating the mismatch between the independently trained duration predictor and TTS model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 34 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

28 of 34 references.

References · 28

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  4. 2024Qwen2-Audio Technical Reportcited by 560
  5. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  6. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  7. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  8. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  9. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  10. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  11. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  12. 2026Qwen3-TTS Technical Reportcited by 118
16 more
  1. 2023CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Maskingcited by 102
  2. 2023LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpuscited by 101
  3. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 98
  4. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 78
  5. 2024SpeechAlign: Aligning Speech Generation to Human Preferencescited by 47
  6. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factorscited by 32
  7. 2024Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimizationcited by 32
  8. 2025Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidancecited by 27
  9. 2026Fish Audio S2 Technical Reportcited by 9
  10. 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 8
  11. 2025WHISTRESS: Enriching Transcriptions with Sentence Stress Detectioncited by 7
  12. 2025StressTest: Can YOUR Speech LM Handle the Stress?cited by 7
  13. 2025Group Relative Policy Optimization for Text-to-Speech with Large Language Modelscited by 7
  14. 2025Fine-Tuning Text-to-Speech Diffusion Models Using Reinforcement Learning with Human Feedbackcited by 4
  15. 2025No Verifiable Reward for Prosody: Toward Preference-Guided Prosody Learning in TTScited by 4
  16. 2026Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmarkcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback
  2. 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization
  3. 2025Multi-Reward GRPO for Stable and Prosodic Single-Codebook TTS LLMs at Scale
  4. 2025EmoSteer-TTS: Fine-Grained and Training-Free Emotion-Controllable Text-to-Speech via Activation Steering
  5. 2025Explore the Reinforcement Learning for the LLM based ASR and TTS system
  6. 2025TTS-CtrlNet: Time varying emotion aligned text-to-speech generation with ControlNet
  7. 2026VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation
  8. 2026FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speech