# MOSPC: MOS Prediction Based on Pairwise Comparison

arXiv: https://arxiv.org/abs/2306.10493 · HTML: https://mlsanity.com/paper/2306.10493/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 18 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion](https://mlsanity.com/paper/2008.12527/) (2020)
- [Generalization Ability of MOS Prediction Networks](https://mlsanity.com/paper/2110.02635/) (2021)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [C-Mixup: Improving Generalization in Regression](https://mlsanity.com/paper/2210.05775/) (2022)
- [MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network](https://mlsanity.com/paper/2103.00110/) (2021)
- [LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech](https://mlsanity.com/paper/2110.09103/) (2021)
- [M3ST: Mix at Three Levels for Speech Translation](https://mlsanity.com/paper/2212.03657/) (2022)
- [Predictions of Subjective Ratings and Spoofing Assessments of Voice Conversion Challenge 2020 Submissions](https://mlsanity.com/paper/2009.03554/) (2020)
- [Fusion of Self-supervised Learned Models for MOS Prediction](https://mlsanity.com/paper/2204.04855/) (2022)
- [DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scores](https://mlsanity.com/paper/2204.03219/) (2022)

## Built on by (most cited first, 7 of 7)

- [The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech](https://mlsanity.com/paper/2409.09305/) (2024)
- [APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speech](https://mlsanity.com/paper/2504.20447/) (2025)
- [WhisQ: Cross-Modal Representation Learning for Text-to-Music MOS Prediction](https://mlsanity.com/paper/2506.05899/) (2025)
- [Efficient Bayesian Inference from Noisy Pairwise Comparisons](https://mlsanity.com/paper/2510.09333/) (2025)
- [AnimeScore: A Preference-Based Dataset and Framework for Evaluating Anime-Like Speech Style](https://mlsanity.com/paper/2603.11482/) (2026)
- [SALF-MOS: Speaker Agnostic Latent Features Downsampled for MOS Prediction](https://mlsanity.com/paper/2506.02082/) (2025)
- [PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets](https://mlsanity.com/paper/2606.19597/) (2026)

## Abstract (arXiv)

As a subjective metric to evaluate the quality of synthesized speech, Mean opinion score~(MOS) usually requires multiple annotators to score the same speech. Such an annotation approach requires a lot of manpower and is also time-consuming. MOS prediction model for automatic evaluation can significantly reduce labor cost. In previous works, it is difficult to accurately rank the quality of speech when the MOS scores are close. However, in practical applications, it is more important to correctly rank the quality of synthesis systems or sentences than simply predicting MOS scores. Meanwhile, as each annotator scores multiple audios during annotation, the score is probably a relative value based on the first or the first few speech scores given by the annotator. Motivated by the above two points, we propose a general framework for MOS prediction based on pair comparison (MOSPC), and we utilize C-Mixup algorithm to enhance the generalization performance of MOSPC. The experiments on BVCC and VCC2018 show that our framework outperforms the baselines on most of the correlation coefficient metrics, especially on the metric KTAU related to quality ranking. And our framework also surpasses the strong baseline in ranking accuracy on each fine-grained segment. These results indicate that our framework contributes to improving the ranking accuracy of speech quality.