mlsanityData to 14 Oct 2026

RLAIF-SPA: Structured AI Feedback for Semantic-Prosodic Alignment in Speech Synthesis

arXiv 2510.14628 ↗First posted 16 Oct 2025cs.CL cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Recent advances in Text-To-Speech (TTS) synthesis have achieved near-human speech quality in neutral speaking styles. However, most existing approaches either depend on costly emotion annotations or optimize surrogate objectives that fail to adequately capture perceptual emotional quality. As a result, the generated speech, while semantically accurate, often lacks expressive and emotionally rich characteristics. To address these limitations, we propose RLAIF-SPA, a novel framework that integrates Reinforcement Learning from AI Feedback (RLAIF) to directly optimize both emotional expressiveness and intelligibility without human supervision. Specifically, RLAIF-SPA incorporates Automatic Speech Recognition (ASR) to provide semantic accuracy feedback, while leveraging structured reward modeling to evaluate prosodic-emotional consistency. RLAIF-SPA enables more precise and nuanced control over expressive speech generation along four structured evaluation dimensions: Structure, Emotion, Speed, and Tone. Extensive experiments on Libri-Speech, MELD, and Mandarin ESD datasets demonstrate consistent gains across clean read speech, conversational dialogue, and emotional speech. On Libri-Speech, RLAIF-SPA consistently outperforms Chat-TTS, achieving a 26.1% reduction in word error rate, a 9.1% improvement in SIM-O, and over 10% gains in human subjective evaluations.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 34 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

25 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024SpeechAlign: Aligning Speech Generation to Human Preferences8 Apr
  2. 2024MusicRL: Aligning Music Generation to Human Preferences6 Feb

Other references · 23

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  6. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 205
  7. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
  8. 2024EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmarkcited by 40
  9. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  10. 2024Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimizationcited by 32
  11. 2019Emotional speech synthesis with rich and granularized controlcited by 28
  12. 2024EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speechcited by 24
11 more
  1. 2024Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedbackcited by 23
  2. 2023A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speechcited by 22
  3. 2023EmoMix: Emotion Mixing via Diffusion Models for Emotional Speech Synthesiscited by 19
  4. 2024Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speechcited by 19
  5. 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Controlcited by 17
  6. 2023EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speechcited by 15
  7. 2023ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Modelscited by 14
  8. 2023MM-TTS: Multi-modal Prompt based Style Transfer for Expressive Text-to-Speech Synthesiscited by 12
  9. 2025SageLM: A Multi-aspect and Explainable Large Language Model for Speech Judgementcited by 7
  10. 2025Group Relative Policy Optimization for Text-to-Speech with Large Language Modelscited by 7
  11. 2024AV-EmoDialog: Chat with Audio-Visual Users Leveraging Emotional Cuescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Do Audio LLMs Listen or Read? Analyzing and Mitigating Paralinguistic Failures with VoxParadoxcited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2026Optimizing Conversational Quality in Spoken Dialogue Systems with Reinforcement Learning from AI Feedback
  2. 2026Bridging What the Model Thinks and How It Speaks: Expressive Speech Generation via Self-Aware Intent-Realization Alignment
  3. 2026WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training
  4. 2025Emotion-Aligned Generation in Diffusion Text to Speech Models via Preference-Guided Optimization
  5. 2026Beyond Speech Captions: Speech-Rewarded Style Planning for Conversational Text-to-Speech
  6. 2024Re-ENACT: Reinforcement Learning for Emotional Speech Generation using Actor-Critic Strategy
  7. 2026SpeechEQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Models
  8. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback