# Investigating Range-Equalizing Bias in Mean Opinion Score Ratings of Synthesized Speech

arXiv: https://arxiv.org/abs/2305.10608 · HTML: https://mlsanity.com/paper/2305.10608/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2023.

## Built on by (most cited first, 13 of 13)

- [The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech](https://mlsanity.com/paper/2409.09305/) (2024)
- [DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Dataset](https://mlsanity.com/paper/2409.08731/) (2024)
- [Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluation](https://mlsanity.com/paper/2411.12719/) (2024)
- [BigVSAN: Enhancing GAN-based Neural Vocoders with Slicing Adversarial Network](https://mlsanity.com/paper/2309.02836/) (2023)
- [The State Of TTS: A Case Study with Human Fooling Rates](https://mlsanity.com/paper/2508.04179/) (2025)
- [TuneJury: An Open Metric for Improving Music Generation Preference Alignment](https://mlsanity.com/paper/2606.17006/) (2026)
- [Towards a GENEA Leaderboard -- an Extended, Living Benchmark for Evaluating and Advancing Conversational Motion Synthesis](https://mlsanity.com/paper/2410.06327/) (2024)
- [The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speech](https://mlsanity.com/paper/2609.13150/) (2026)
- [Towards Reliable Human Evaluations in Gesture Generation: Insights from a Community-Driven State-of-the-Art Benchmark](https://mlsanity.com/paper/2511.01233/) (2025)
- [Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languages](https://mlsanity.com/paper/2604.21481/) (2026)
- [Automatic design optimization of preference-based subjective evaluation with online learning in crowdsourcing environment](https://mlsanity.com/paper/2403.06100/) (2024)
- [SA-SSL-MOS: Self-supervised Learning MOS Prediction with Spectral Augmentation for Generalized Multi-Rate Speech Assessment](https://mlsanity.com/paper/2602.14785/) (2026)
- [Voice Mapping of Text-to-Speech Systems: A Metric-Based Approach for Voice Quality Assessment](https://mlsanity.com/paper/2605.00861/) (2026)

## Abstract (arXiv)

Mean Opinion Score (MOS) is a popular measure for evaluating synthesized speech. However, the scores obtained in MOS tests are heavily dependent upon many contextual factors. One such factor is the overall range of quality of the samples presented in the test -- listeners tend to try to use the entire range of scoring options available to them regardless of this, a phenomenon which is known as range-equalizing bias. In this paper, we systematically investigate the effects of range-equalizing bias on MOS tests for synthesized speech by conducting a series of listening tests in which we progressively "zoom in" on a smaller number of systems in the higher-quality range. This allows us to better understand and quantify the effects of range-equalizing bias in MOS tests.