# A Comparison of Deep Learning MOS Predictors for Speech Synthesis Quality

arXiv: https://arxiv.org/abs/2204.02249 · HTML: https://mlsanity.com/paper/2204.02249/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Apr 2022.

## Built on by (most cited first, 5 of 5)

- [SQuId: Measuring Speech Naturalness in Many Languages](https://mlsanity.com/paper/2210.06324/) (2022)
- [Evaluation of Speech Representations for MOS prediction](https://mlsanity.com/paper/2306.09979/) (2023)
- [The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speech](https://mlsanity.com/paper/2609.13150/) (2026)
- [Developing vocal system impaired patient-aimed voice quality assessment approach using ASR representation-included multiple features](https://mlsanity.com/paper/2408.12279/) (2024)
- [JSQA: Speech Quality Assessment with Perceptually-Inspired Contrastive Pretraining Based on JND Audio Pairs](https://mlsanity.com/paper/2507.11636/) (2025)

## Abstract (arXiv)

Speech synthesis quality prediction has made remarkable progress with the development of supervised and self-supervised learning (SSL) MOS predictors but some aspects related to the data are still unclear and require further study. In this paper, we evaluate several MOS predictors based on wav2vec 2.0 and the NISQA speech quality prediction model to explore the role of the training data, the influence of the system type, and the role of cross-domain features in SSL models. Our evaluation is based on the VoiceMOS challenge dataset. Results show that SSL-based models show the highest correlation and lowest mean squared error compared to supervised models. The key point of this study is that benchmarking the statistical performance of MOS predictors alone is not sufficient to rank models since potential issues hidden in the data could bias the evaluated performances.