mlsanityData to 14 Oct 2026

FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speech

arXiv 2606.23190 ↗First posted 22 Jun 2026eess.AS cs.SDcited by 5

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Existing Reinforcement Learning (RL) research for Text-to-Speech (TTS) focuses on large language models (LLMs), leaving Flow-Matching (FM) under-explored. We present FlowTTS-GRPO, an online RL framework for FM-based TTS. By converting ordinary differential equation (ODE) trajectories into stochastic differential equation (SDE) paths, our method enables direct fine-tuning of open-source FM models without auxiliary models. We show that a weighted reward combination converges faster than a probabilistic scheme, and identify three practical optimizations: omitting classifier-free guidance (CFG) during training accelerates convergence; synthesizing hard cases improves robustness; and applying RL to the FM component enhances audio-detail metrics. Experiments on CosyVoice 3.0 and F5-TTS demonstrate objective and subjective preference gains in speaker similarity and perceptual quality, with F5-TTS also improving intelligibility.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 41 of its 42 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

41 of 42 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026FlowSE-GRPO: Training Flow Matching Speech Enhancement via Online Reinforcement Learning23 Jan
  2. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training23 May
  3. 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization3 Apr
  4. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models13 Dec
  5. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct

Other references · 36

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  7. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  8. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  9. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  10. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  11. 2022High Fidelity Neural Audio Compressioncited by 687
  12. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
24 more
  1. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  2. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  3. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  4. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
  5. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 178
  6. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 172
  7. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  8. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  9. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  10. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  11. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 90
  12. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 78
  13. 2023LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTcited by 78
  14. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 70
  15. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 43
  16. 2023An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verificationcited by 42
  17. 2024WenetSpeech4TTS: A 12,800-hour Mandarin TTS Corpus for Large Speech Generation Model Benchmarkcited by 40
  18. 2024Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimizationcited by 32
  19. 2025Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidancecited by 27
  20. 2024Preference Alignment Improves Language Model-Based TTScited by 21
  21. 2025Differentiable Reward Optimization for LLM based TTS systemcited by 15
  22. 2026PromptRL: Prompt Matters in RL for Flow-Based Image Generationcited by 12
  23. 2025Group Relative Policy Optimization for Text-to-Speech with Large Language Modelscited by 7
  24. 2025RRPO: Robust Reward Policy Optimization for LLM-based Emotional TTScited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Renderingcited by 4
  2. 2026AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editingcited by 1
  3. 2026NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speechcited by 0
  4. 2026Can LLM Agents Automate Reinforcement Learning for Text-to-Speech?cited by 0
  5. 2026JoyAI-Voice 2.0: A Full-Continuous Autoregressive Speech Generation Model with Semantic-Acoustic Joint Representationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GROW: Group-Relative Advantage-Weighted On-Policy Reinforcement Learning of Autoregressive-Diffusion Text-to-Speech model
  2. 2025Explore the Reinforcement Learning for the LLM based ASR and TTS system
  3. 2026Resonate: Reinforcing Text-to-Audio Generation via Online Feedback from Large Audio Language Models ↗
  4. 2025Multi-Reward GRPO for Stable and Prosodic Single-Codebook TTS LLMs at Scale
  5. 2025Towards Flow-Matching-based TTS without Classifier-Free Guidance
  6. 2026VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation
  7. 2026Faster and Better Alignment for Flow Matching Models via Step-aware Advantages
  8. 2026Investigating Group Relative Policy Optimization for Diffusion Transformer based Text-to-Audio Generation