mlsanityData to 7 Oct 2026

Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization

arXiv 2407.02243 ↗First posted 2 Jul 2024cs.CL cs.SD eess.AScited by 10

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

In this paper, we propose reverse inference optimization (RIO), a simple and effective method designed to enhance the robustness of autoregressive-model-based zero-shot text-to-speech (TTS) systems using reinforcement learning from human feedback (RLHF). To assess the quality of speech produced by the TTS system without human annotations, RIO introduces a novel concept termed as reverse inference based on the Bayesian principle, which suggests that a high-quality generated speech should be able to be used as a prompt for subsequent generation using the same TTS model. By leveraging reverse inference as the standard to select exemplars used in RLHF from the speech samples generated by the TTS system itself, RIO steers the subsequent optimization towards a direction of enhancing the TTS robustness. The RIO framework, comprising sampling, automatic annotating, and learning, obviates the need for a reward model or pairwise preference data, and significantly improves the stability of zero-shot TTS performance by reducing the discrepancies between training and inference conditions. Our experimental results verify that RIO can effectively improve both subjective and objective metrics, including mean opinion scores, word error rates, and speaker similarity. Remarkably, RIO can also diminish the incidence of bad outputs to nearly zero percent, rivalling the robustness when using ground-truth speech as the prompt.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 45 of its 51 references that are in the map. First posted on arXiv 2 Jul 2024.

Builds on

45 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback2 Jun
  2. 2024SpeechAlign: Aligning Speech Generation to Human Preferences8 Apr

Other references · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  10. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  11. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  12. 2022High Fidelity Neural Audio Compressioncited by 673
31 more
  1. 2024Self-Rewarding Language Modelscited by 654
  2. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  3. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  4. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  5. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  6. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  7. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  8. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  9. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  10. 2023Generating Images with Multimodal Language Modelscited by 259
  11. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  12. 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 213
  13. 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 167
  14. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
  15. 2024Token-level Direct Preference Optimizationcited by 135
  16. 2024Direct Preference Optimization with an Offsetcited by 122
  17. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 118
  18. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 90
  19. 2023VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translationcited by 87
  20. 2023SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Modelscited by 85
  21. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
  22. 2021Generalization Ability of MOS Prediction Networkscited by 69
  23. 2024Natural language guidance of high-fidelity text-to-speech with synthetic annotationscited by 66
  24. 2020Text Generation by Learning from Demonstrationscited by 64
  25. 2023Better speech synthesis through scalingcited by 63
  26. 2023TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Modelscited by 48
  27. 2024Enhancing LLM Safety via Constrained Direct Preference Optimizationcited by 47
  28. 2023PromptStyle: Controllable Style Transfer for Text-to-Speech with Natural Language Descriptionscited by 33
  29. 2023RepCodec: A Speech Representation Codec for Speech Tokenizationcited by 27
  30. 2024RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesiscited by 26
  31. 2023Context-Aware Selective Label Smoothing for Calibrating Sequence Recognition Modelcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 1.

10 papers.

  1. 2024Preference Alignment Improves Language Model-Based TTScited by 21
  2. 2025Differentiable Reward Optimization for LLM based TTS systemcited by 15
  3. 2024Takin: A Cohort of Superior Quality Zero-shot Speech Generation Modelscited by 7
  4. 2025Robust and Efficient Autoregressive Speech Synthesis with Dynamic Chunk-wise Prediction Policycited by 4
  5. 2025Direct Preference Optimization for Speech Autoregressive Diffusion Modelscited by 4
  6. 2025Multi-Reward GRPO for Stable and Prosodic Single-Codebook TTS LLMs at Scalecited by 3
  7. 2025MARS6: A Small and Robust Hierarchical-Codec Text-to-Speech Modelcited by 1
  8. 2025Vox-Evaluator: Enhancing Stability and Fidelity for Zero-shot TTS with A Multi-Level Evaluatorcited by 1
  9. 2025Speech Recognition Model Improves Text-to-Speech Synthesis using Fine-Grained Rewardcited by 1
  10. 2026Best-of-N TTS Evaluation is Confounded by ASR Family Alignmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026VoiceTTA: Enhancing Zero-Shot Text-to-Speech via Reinforcement Learning-Based Test-Time Adaptation
  2. 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment
  3. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  4. 2025CLEAR: Continuous Latent Autoregressive Modeling for High-quality and Low-latency Speech Synthesis
  5. 2026DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech
  6. 2026Tacit-TTS: From Autoregressive Decoding to Masked Prediction for Efficient Transcript-Free Voice Cloning
  7. 2024Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation
  8. 2026Boosting ASR Robustness via Test-Time Reinforcement Learning with Audio-Text Semantic Rewards