mlsanityData to 14 Oct 2026

Emotion-Aligned Generation in Diffusion Text to Speech Models via Preference-Guided Optimization

arXiv 2509.25416 ↗First posted 29 Sep 2025cs.CL cs.AI cs.SDcited by 2

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Emotional text-to-speech seeks to convey affect while preserving intelligibility and prosody, yet existing methods rely on coarse labels or proxy classifiers and receive only utterance-level feedback. We introduce Emotion-Aware Stepwise Preference Optimization (EASPO), a post-training framework that aligns diffusion TTS with fine-grained emotional preferences at intermediate denoising steps. Central to our approach is EASPM, a time-conditioned model that scores noisy intermediate speech states and enables automatic preference pair construction. EASPO optimizes generation to match these stepwise preferences, enabling controllable emotional shaping. Experiments show superior performance over existing methods in both expressiveness and naturalness.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 17 of its 19 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

17 of 19 references.

References · 17

  1. 2020Denoising Diffusion Implicit Modelscited by 9,781
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  4. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  5. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  7. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  8. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  9. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 192
  10. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 110
  11. 2024PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Modelscited by 42
  12. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
5 more
  1. 2025EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Promptingcited by 34
  2. 2024Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimizationcited by 32
  3. 2021Fine-grained style control in Transformer-based Text-to-speech Synthesiscited by 15
  4. 2023EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speechcited by 15
  5. 2025CLEP-DG: Contrastive Learning for Speech Emotion Domain Generalization via Soft Prompt Tuningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluationcited by 1
  2. 2026Speaker Identity in Non-Verbal Vocalizations: Conditional Distillation and Mixture of Experts Approachcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Models
  2. 2026HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speech
  3. 2026Learning to Prefer Reliably: Error-Augmented Emotion Preference Optimization with Calibrated Fusion
  4. 2026Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech
  5. 2026Tracing a Sparse Emotion-Control Circuit in LLM-Based Text-to-Speech
  6. 2024AlignCap: Aligning Speech Emotion Captioning to Human Preferences
  7. 2026EmoRES-TTS: Residual-Enhanced Vector Steering for Emotional Speech Generation
  8. 2026Acoustic and Semantic Modeling of Emotion in Spoken Language ↗