# MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Models

arXiv: https://arxiv.org/abs/2411.03715 · HTML: https://mlsanity.com/paper/2411.03715/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2024.

## Built on by (most cited first, 12 of 12)

- [ESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems](https://mlsanity.com/paper/2503.08533/) (2025)
- [SingMOS-Pro: An Comprehensive Benchmark for Singing Quality Assessment](https://mlsanity.com/paper/2510.01812/) (2025)
- [SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis](https://mlsanity.com/paper/2602.07803/) (2026)
- [APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speech](https://mlsanity.com/paper/2504.20447/) (2025)
- [The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speech](https://mlsanity.com/paper/2609.13150/) (2026)
- [Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Tasks](https://mlsanity.com/paper/2601.12205/) (2026)
- [VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing](https://mlsanity.com/paper/2605.06765/) (2026)
- [Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditions](https://mlsanity.com/paper/2509.24457/) (2025)
- [PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets](https://mlsanity.com/paper/2606.19597/) (2026)
- [Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations](https://mlsanity.com/paper/2606.19951/) (2026)
- [PASQA: Pitch-Accent-Focused Speech Quality Assessment Model Trained on Synthetic Speech with Accent Errors](https://mlsanity.com/paper/2606.20137/) (2026)
- [A Comprehensive Objective Evaluation of Modern Text-to-Speech for Turkish Using Speech Quality Assessment Models](https://mlsanity.com/paper/2610.06057/) (2026)

## Abstract (arXiv)

In this paper, we study the task of subjective speech quality assessment (SSQA), which refers to predicting the perceptual quality of speech. Owing to the development of deep neural network models, SSQA has greatly advanced and has been widely applied in scientific papers to evaluate speech generation systems. Nonetheless, the insufficient out-of-domain (OOD) generalization ability of current SSQA models is underexplored and often overlooked by researchers. To study this problem systematically, we present MOS-Bench, a diverse SSQA dataset collection that currently contains 8 training sets and 17 test sets. Through extensive experiments, we first highlight the OOD generalization challenges of existing models. We then evaluate the efficacy of multiple-dataset training, comparing straightforward data pooling against AlignNet, an existing domain-aware method. We demonstrate that pooling multiple training sets provides a simple yet effective solution, and variation in the data is a key factor for robust generalization beyond training data size.