# Group Relative Policy Optimization for Text-to-Speech with Large Language Models

arXiv: https://arxiv.org/abs/2509.18798 · HTML: https://mlsanity.com/paper/2509.18798/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2025.

## Built on by (most cited first, 7 of 7)

- [FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speech](https://mlsanity.com/paper/2606.23190/) (2026)
- [GSRM: Generative Speech Reward Model for Speech RLHF](https://mlsanity.com/paper/2602.13891/) (2026)
- [RLAIF-SPA: Structured AI Feedback for Semantic-Prosodic Alignment in Speech Synthesis](https://mlsanity.com/paper/2510.14628/) (2025)
- [Investigating Group Relative Policy Optimization for Diffusion Transformer based Text-to-Audio Generation](https://mlsanity.com/paper/2603.01565/) (2026)
- [Imitation Learning for Elder-Facing Speech Synthesis](https://mlsanity.com/paper/2606.21053/) (2026)
- [EmphTTS: an emphasis-control TTS with reinforcement learning](https://mlsanity.com/paper/2609.27599/) (2026)
- [NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speech](https://mlsanity.com/paper/2609.31892/) (2026)

## Abstract (arXiv)

This paper proposes a GRPO-based approach to enhance the performance of large language model (LLM)-based text-to-speech (TTS) models by deriving rewards from an off-the-shelf automatic speech recognition (ASR) model. Compared to previous reinforcement learning methods for LLM-based TTS, our method requires no dedicated model for reward computation or training. Moreover, we design a composite reward function that combines character error rate (CER) with negative log-likelihood (NLL) obtained from the ASR model, providing more informative and accurate reward signals. We apply GRPO fine-tuning to pre-trained LLM-based TTS models and evaluate their zero-shot TTS performance. Experimental results show that the proposed method substantially improves both the intelligibility and naturalness of synthesized speech. Ablation studies and further analyses confirm the effectiveness of integrating the two reward components.