mlsanityData to 14 Oct 2026

A Comprehensive Objective Evaluation of Modern Text-to-Speech for Turkish Using Speech Quality Assessment Models

arXiv 2610.06057 ↗First posted 5 Oct 2026cs.SD cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern text-to-speech (TTS) systems can clone a target speaker from a short reference clip or be fine-tuned on a target voice, yet their behaviour on morphologically rich, lower-resource languages such as Turkish remain under-characterised. We present a systematic benchmark of four contemporary systems (Chatterbox, CosyVoice, OmniVoice, and VoxCPM2) evaluated across fine-tuned and zero-shot configurations, contrasted with a conventional VITS baseline and anchored to natural gold speech. Each configuration is scored with eighteen complementary objective metrics spanning learned naturalness predictors (UTMOS v2, DNSMOS-Pro, SCOREQ, WhisQA, AudioBox-PQ, NatScore, SpeechLMScore), intelligibility and signal-quality estimators (SQUIM PESQ/SI-SDR/STOI, Brouhaha), speaker similarity, distributional fidelity (TTSDS) and low-level acoustic descriptors. We further analyse how quality varies with utterance length and quantify long-form temporal consistency through speaker-identity and naturalness drift over chunked utterances. We release our evaluation code to support reproducible TTS evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 27 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

21 of 27 references.

References · 21

  1. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  2. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  3. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  4. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
  5. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
  6. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
  7. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 167
  8. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  9. 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
  10. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
  11. 2023TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudiocited by 59
  12. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Modelscited by 25
9 more
  1. 2024SCOREQ: Speech Quality Assessment with Contrastive Regressioncited by 24
  2. 2022Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimationcited by 23
  3. 2022SpeechLMScore: Evaluating speech generation using speech language modelcited by 20
  4. 2026VoxCPM2 Technical Reportcited by 17
  5. 2024MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Modelscited by 12
  6. 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 8
  7. 2025From Scores to Preferences: Redefining MOS Benchmarking for Speech Quality Reward Modelingcited by 5
  8. 2026UrgentMOS: Unified Multi-Metric and Preference Learning for Robust Speech Quality Assessmentcited by 4
  9. 2025WhiSQA: Non-Intrusive Speech Quality Prediction Using Whisper Encoder Featurescited by 2

Built on by

Not cited yet.