# Generalization Ability of MOS Prediction Networks

arXiv: https://arxiv.org/abs/2110.02635 · HTML: https://mlsanity.com/paper/2110.02635/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2021.

## Built on by (most cited first, 30 of 69)

- [The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech](https://mlsanity.com/paper/2409.09305/) (2024)
- [Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks](https://mlsanity.com/paper/2309.07937/) (2023)
- [Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features](https://mlsanity.com/paper/2111.02363/) (2021)
- [The Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2306.14422/) (2023)
- [Audio Large Language Models Can Be Descriptive Speech Quality Evaluators](https://mlsanity.com/paper/2501.17202/) (2025)
- [LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech](https://mlsanity.com/paper/2110.09103/) (2021)
- [Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback](https://mlsanity.com/paper/2411.01834/) (2024)
- [SpeechLMScore: Evaluating speech generation using speech language model](https://mlsanity.com/paper/2212.04559/) (2022)
- [Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback](https://mlsanity.com/paper/2406.00654/) (2024)
- [SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis](https://mlsanity.com/paper/2204.03040/) (2022)
- [Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation](https://mlsanity.com/paper/2409.16644/) (2024)
- [Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization](https://mlsanity.com/paper/2407.02243/) (2024)
- [Fusion of Self-supervised Learned Models for MOS Prediction](https://mlsanity.com/paper/2204.04855/) (2022)
- [SQuId: Measuring Speech Naturalness in Many Languages](https://mlsanity.com/paper/2210.06324/) (2022)
- [FELLE: Autoregressive Speech Synthesis with Token-Wise Coarse-to-Fine Flow Matching](https://mlsanity.com/paper/2502.11128/) (2025)
- [InQSS: a speech intelligibility and quality assessment model using a multi-task learning network](https://mlsanity.com/paper/2111.02585/) (2021)
- [DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scores](https://mlsanity.com/paper/2204.03219/) (2022)
- [MOSPC: MOS Prediction Based on Pairwise Comparison](https://mlsanity.com/paper/2306.10493/) (2023)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [SongBench: A Fine-Grained Multi-Aspect Benchmark for Song Quality Assessment](https://mlsanity.com/paper/2604.25937/) (2026)
- [MTI-Net: A Multi-Target Speech Intelligibility Prediction Model](https://mlsanity.com/paper/2204.03310/) (2022)
- [Malacopula: adversarial automatic speaker verification attacks using a neural-based generalised Hammerstein model](https://mlsanity.com/paper/2408.09300/) (2024)
- [On the Use of Self-Supervised Speech Representations in Spontaneous Speech Synthesis](https://mlsanity.com/paper/2307.05132/) (2023)
- [Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata](https://mlsanity.com/paper/2309.09548/) (2023)
- [Robust One-step Speech Enhancement via Consistency Distillation](https://mlsanity.com/paper/2507.05688/) (2025)
- [A Comparative Study of Self-Supervised Speech Representations in Read and Spontaneous TTS](https://mlsanity.com/paper/2303.02719/) (2023)
- [Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages](https://mlsanity.com/paper/2305.19396/) (2023)
- [Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals using Self Supervised Speech Representations](https://mlsanity.com/paper/2307.13423/) (2023)
- [Evaluating Speech Synthesis by Training Recognizers on Synthetic Speech](https://mlsanity.com/paper/2310.00706/) (2023)
- [SpeechComposer: Unifying Multiple Speech Tasks with Prompt Composition](https://mlsanity.com/paper/2401.18045/) (2024)

## Abstract (arXiv)

Automatic methods to predict listener opinions of synthesized speech remain elusive since listeners, systems being evaluated, characteristics of the speech, and even the instructions given and the rating scale all vary from test to test. While automatic predictors for metrics such as mean opinion score (MOS) can achieve high prediction accuracy on samples from the same test, they typically fail to generalize well to new listening test contexts. In this paper, using a variety of networks for MOS prediction including MOSNet and self-supervised speech models such as wav2vec2, we investigate their performance on data from different listening tests in both zero-shot and fine-tuned settings. We find that wav2vec2 models fine-tuned for MOS prediction have good generalization capability to out-of-domain data even for the most challenging case of utterance-level predictions in the zero-shot setting, and that fine-tuning to in-domain data can improve predictions. We also observe that unseen systems are especially challenging for MOS prediction models.