# When Synthetic Speech Is All You Have: Better Call GRPO

arXiv: https://arxiv.org/abs/2607.08409 · HTML: https://mlsanity.com/paper/2607.08409/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 36 references that are in the map.
- First posted on arXiv 9 Jul 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 28 of 28)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [AudioPaLM: A Large Language Model That Can Speak and Listen](https://mlsanity.com/paper/2306.12925/) (2023)
- [Qwen3-ASR Technical Report](https://mlsanity.com/paper/2601.21337/) (2026)
- [An Embarrassingly Simple Approach for LLM with Strong ASR Capacity](https://mlsanity.com/paper/2402.08846/) (2024)
- [Speech Recognition with Augmented Synthesized Speech](https://mlsanity.com/paper/1909.11699/) (2019)
- [Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems](https://mlsanity.com/paper/1912.09257/) (2019)
- [Building and Evaluation of a Real Room Impulse Response Dataset](https://mlsanity.com/paper/1811.06795/) (2018)
- [Large-Scale Self- and Semi-Supervised Learning for Speech Translation](https://mlsanity.com/paper/2104.06678/) (2021)
- [Task Arithmetic can Mitigate Synthetic-to-Real Gap in Automatic Speech Recognition](https://mlsanity.com/paper/2406.02925/) (2024)
- [Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gap](https://mlsanity.com/paper/2410.16726/) (2024)
- [Group Relative Policy Optimization for Speech Recognition](https://mlsanity.com/paper/2509.01939/) (2025)
- [Text-only adaptation in LLM-based ASR through text denoising](https://mlsanity.com/paper/2601.20900/) (2026)
- [A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Data](https://mlsanity.com/paper/2506.11130/) (2025)
- [Value Alignment from Unstructured Text](https://mlsanity.com/paper/2408.10392/) (2024)
- [The State Of TTS: A Case Study with Human Fooling Rates](https://mlsanity.com/paper/2508.04179/) (2025)
- [Towards Improved Speech Recognition through Optimized Synthetic Data Generation](https://mlsanity.com/paper/2508.21631/) (2025)
- [Distilling Conversations: Abstract Compression of Conversational Audio Context for LLM-based ASR](https://mlsanity.com/paper/2603.26246/) (2026)
- [How to Leverage Synthetic Speech for LLM-Based ASR Systems?](https://mlsanity.com/paper/2606.29031/) (2026)

## Abstract (arXiv)

LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute. Yet synthetic speech stays acoustically mismatched with real recordings, and work on this gap has stayed within supervised fine-tuning (SFT). We instead turn to reinforcement learning, and show that Group Relative Policy Optimization (GRPO) extracts far more from the same synthetic speech than SFT. Synthetic-only adaptation of the model with GRPO, a critic-free method rewarding low-WER hypotheses, reduces WER by 40% relative to SFT (36.71%→22.09%), and an SFT-then-GRPO combination pushes this further to 45%. We trace the gain to behavior rather than representation: GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, leaving early-layer representations intact. When synthetic speech is the main resource, reinforcement learning should be preferred over supervised fine-tuning.