A Dataset for Automatic Assessment of TTS Quality in Spanish
On the map: Speech and audio language models → Vocoders and speech synthesis
Abstract (arXiv)
This work addresses the development of a database for the automatic assessment of text-to-speech (TTS) systems in Spanish, aiming to improve the accuracy of naturalness prediction models. The dataset consists of 4,326 audio samples from 52 different TTS systems and human voices and is, up to our knowledge, the first of its kind in Spanish. To label the audios, a subjective test was designed based on the ITU-T Rec. P.807 standard and completed by 92 participants. Furthermore, the utility of the collected dataset was validated by training automatic naturalness prediction systems. We explored two approaches: fine-tuning an existing model originally trained for English, and training small downstream networks on top of frozen self-supervised speech models. Our models achieve a mean absolute error of 0.8 on a five-point MOS scale. Further analysis demonstrates the quality and diversity of the developed dataset, and its potential to advance TTS research in Spanish.
Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 10 of its 25 references that are in the map. First posted on arXiv 2 Jul 2025.
Builds on
10 of 25 references.
Closest ancestors · 2
Same direction, newest first
- 2023DCTTS: Discrete Diffusion Model with Contrastive Learning for Text-to-speech Generation13 Sep
- 2021Deep Learning Based Assessment of Synthetic Speech Naturalness23 Apr
Other references · 8
- 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 270
- 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 204
- 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 184
- 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 97
- 2021NORESQA: A Framework for Speech Quality Assessment using Non-Matching Referencescited by 23
- 2020On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognitioncited by 21
- 2020Recognizing More Emotions with Less Data Using Self-supervised Transfer Learningcited by 8
- 2021Bias-Aware Loss for Training Image and Speech Quality Prediction Models from Multiple Datasetscited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions
- 2026Domain-Specific Evaluation of Text-to-Speech Systems: A Multi-Metric Benchmarking Study
- 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems
- 2021Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech
- 2022Can we use Common Voice to train a Multi-Speaker TTS system?
- 2024Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gap
- 2024TTSDS -- Text-to-Speech Distribution Score
- 2026A Comprehensive Objective Evaluation of Modern Text-to-Speech for Turkish Using Speech Quality Assessment Models