# Rho-Perfect: Correlation Ceiling For Subjective Evaluation Datasets

arXiv: https://arxiv.org/abs/2602.08552 · HTML: https://mlsanity.com/paper/2602.08552/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 19 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 3 of 3)

- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [The VoiceMOS Challenge 2022](https://mlsanity.com/paper/2203.11389/) (2022)

## Built on by (most cited first, 1 of 1)

- [Perceptual Quality Loss or Loss of Perceptual Quality?](https://mlsanity.com/paper/2609.35054/) (2026)

## Abstract (arXiv)

Subjective ratings contain inherent noise that limits the model-human correlation, but this reliability issue is rarely quantified. In this paper, we present ρ-Perfect, a practical estimation of the highest achievable correlation of a model on subjectively rated datasets. We define ρ-Perfect to be the correlation between a perfect predictor and human ratings, and derive an estimate of the value based on heteroscedastic noise scenarios, a common occurrence in subjectively rated datasets. We show that ρ-Perfect squared estimates test-retest correlation and use this to validate the estimate. We demonstrate the use of ρ-Perfect on a speech quality dataset and show how the measure can distinguish between model limitations and data quality issues.