mlsanityData to 7 Oct 2026

When Synthetic Speech Is All You Have: Better Call GRPO

arXiv 2607.08409 ↗First posted 9 Jul 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute. Yet synthetic speech stays acoustically mismatched with real recordings, and work on this gap has stayed within supervised fine-tuning (SFT). We instead turn to reinforcement learning, and show that Group Relative Policy Optimization (GRPO) extracts far more from the same synthetic speech than SFT. Synthetic-only adaptation of the model with GRPO, a critic-free method rewarding low-WER hypotheses, reduces WER by 40% relative to SFT (36.71%→22.09%), and an SFT-then-GRPO combination pushes this further to 45%. We trace the gain to behavior rather than representation: GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, leaving early-layer representations intact. When synthetic speech is the main resource, reinforcement learning should be preferred over supervised fine-tuning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 36 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

28 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May

Other references · 27

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2023Instruction Tuning with GPT-4cited by 676
  6. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 591
  7. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  8. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  9. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  10. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  11. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  12. 2026Qwen3-ASR Technical Reportcited by 105
15 more
  1. 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacitycited by 86
  2. 2019Speech Recognition with Augmented Synthesized Speechcited by 60
  3. 2019Generating Synthetic Audio Data for Attention-Based Speech Recognition Systemscited by 46
  4. 2018Building and Evaluation of a Real Room Impulse Response Datasetcited by 43
  5. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  6. 2024Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognitioncited by 14
  7. 2024Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gapcited by 11
  8. 2025Group Relative Policy Optimization for Speech Recognitioncited by 8
  9. 2026Text-only adaptation in LLM-based ASR through text denoisingcited by 8
  10. 2025A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Datacited by 7
  11. 2024Value Alignment from Unstructured Textcited by 4
  12. 2025The State Of TTS: A Case Study with Human Fooling Ratescited by 4
  13. 2025Towards Improved Speech Recognition through Optimized Synthetic Data Generationcited by 3
  14. 2026Distilling Conversations: Abstract Compression of Conversational Audio Context for LLM-based ASRcited by 2
  15. 2026How to Leverage Synthetic Speech for LLM-Based ASR Systems?cited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FlowSE-GRPO: Training Flow Matching Speech Enhancement via Online Reinforcement Learning
  2. 2024Beyond Oversmoothing: Evaluating DDPM and MSE for Scalable Speech Synthesis in ASR
  3. 2025Advancing Speech Understanding in Speech-Aware Language Models with GRPO
  4. 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization
  5. 2025GRPO-RM: Fine-Tuning Representation Models via GRPO-Driven Reinforcement Learning
  6. 2026Advances in GRPO for Generation Models: A Survey
  7. 2026Transcribe, Translate, and Optimize: Joint Reward Learning for Speech Translation
  8. 2025Explore the Reinforcement Learning for the LLM based ASR and TTS system