# Transformation of Mean Opinion Scores to Avoid Misleading of Ranked based Statistical Techniques

arXiv: https://arxiv.org/abs/2004.11490 · HTML: https://mlsanity.com/paper/2004.11490/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2020.

## Built on by (most cited first, 3 of 3)

- [Bias-Aware Loss for Training Image and Speech Quality Prediction Models from Multiple Datasets](https://mlsanity.com/paper/2104.10217/) (2021)
- [Full Reference Video Quality Assessment for Machine Learning-Based Video Codecs](https://mlsanity.com/paper/2309.00769/) (2023)
- [ICME 2025 Grand Challenge on Video Super-Resolution for Video Conferencing](https://mlsanity.com/paper/2506.12269/) (2025)

## Abstract (arXiv)

The rank correlation coefficients and the ranked-based statistical tests (as a subset of non-parametric techniques) might be misleading when they are applied to subjectively collected opinion scores. Those techniques assume that the data is measured at least at an ordinal level and define a sequence of scores to represent a tied rank when they have precisely an equal numeric value. In this paper, we show that the definition of tied rank, as mentioned above, is not suitable for Mean Opinion Scores (MOS) and might be misleading conclusions of rank-based statistical techniques. Furthermore, we introduce a method to overcome this issue by transforming the MOS values considering their 95% Confidence Intervals. The rank correlation coefficients and ranked-based statistical tests can then be safely applied to the transformed values. We also provide open-source software packages in different programming languages to utilize the application of our transformation method in the quality of experience domain.