mlsanityData to 14 Oct 2026

Vox-Evaluator: Enhancing Stability and Fidelity for Zero-shot TTS with A Multi-Level Evaluator

arXiv 2510.20210 ↗First posted 23 Oct 2025cs.SDcited by 1

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recent advances in zero-shot text-to-speech (TTS), driven by language models, diffusion models and masked generation, have achieved impressive naturalness in speech synthesis. Nevertheless, stability and fidelity remain key challenges, manifesting as mispronunciations, audible noise, and quality degradation. To address these issues, we introduce Vox-Evaluator, a multi-level evaluator designed to guide the correction of erroneous speech segments and preference alignment for TTS systems. It is capable of identifying the temporal boundaries of erroneous segments and providing a holistic quality assessment of the generated speech. Specifically, to refine erroneous segments and enhance the robustness of the zero-shot TTS model, we propose to automatically identify acoustic errors with the evaluator, mask the erroneous segments, and finally regenerate speech conditioning on the correct portions. In addition, the fine-gained information obtained from Vox-Evaluator can guide the preference alignment for TTS model, thereby reducing the bad cases in speech synthesis. Due to the lack of suitable training datasets for the Vox-Evaluator, we also constructed a synthesized text-speech dataset annotated with fine-grained pronunciation errors or audio quality issues. The experimental results demonstrate the effectiveness of the proposed Vox-Evaluator in enhancing the stability and fidelity of TTS systems through the speech correction mechanism and preference optimization. The demos are shown.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 37 of its 49 references that are in the map. First posted on arXiv 23 Oct 2025.

Builds on

37 of 49 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization3 Apr
  2. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  3. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr

Other references · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  8. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  9. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  10. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  11. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  12. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
22 more
  1. 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 169
  2. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 167
  3. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  4. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  5. 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
  6. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 122
  7. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 120
  8. 2020CTC-Segmentation of Large Corpora for German End-to-end Speech Recognitioncited by 76
  9. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 75
  10. 2022Unified Speech-Text Pre-training for Speech Translation and Recognitioncited by 64
  11. 2022SpeechLM: Enhanced Speech Pre-Training with Unpaired Textual Datacited by 49
  12. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 49
  13. 2024SpeechAlign: Aligning Speech Generation to Human Preferencescited by 47
  14. 2025Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidancecited by 27
  15. 2024Preference Alignment Improves Language Model-Based TTScited by 21
  16. 2024Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedbackcited by 20
  17. 2025MoonCast: High-Quality Zero-Shot Podcast Generationcited by 18
  18. 2025Advancing Zero-shot Text-to-Speech Intelligibility across Diverse Domains via Preference Alignmentcited by 18
  19. 2021Context-Aware Prosody Correction for Text-Based Speech Editingcited by 13
  20. 2024Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimizationcited by 11
  21. 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 8
  22. 2025MultiActor-Audiobook: Zero-Shot Audiobook Generation with Faces and Voices of Multiple Speakerscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Tone on a Budget: A Reference-Free Metric for Lexical Tone in Massively Multilingual Text-to-Speechcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation
  2. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis
  3. 2026Training a Compact, Strong, and Realistic Text-to-Speech Model
  4. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  5. 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment
  6. 2024An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS
  7. 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesis
  8. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias