# PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets

arXiv: https://arxiv.org/abs/2606.19597 · HTML: https://mlsanity.com/paper/2606.19597/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 16 of 16)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis](https://mlsanity.com/paper/2204.03040/) (2022)
- [The VoiceMOS Challenge 2024: Beyond Speech Quality Prediction](https://mlsanity.com/paper/2409.07001/) (2024)
- [SpeechJudge: Towards Human-Level Judgment for Speech Naturalness](https://mlsanity.com/paper/2511.07931/) (2025)
- [SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation](https://mlsanity.com/paper/2510.14664/) (2025)
- [SESQA: semi-supervised learning for speech quality assessment](https://mlsanity.com/paper/2010.00368/) (2020)
- [NORESQA: A Framework for Speech Quality Assessment using Non-Matching References](https://mlsanity.com/paper/2109.08125/) (2021)
- [MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Models](https://mlsanity.com/paper/2411.03715/) (2024)
- [A Pyramid Recurrent Network for Predicting Crowdsourced Speech-Quality Ratings of Real-World Signals](https://mlsanity.com/paper/2007.15797/) (2020)
- [MOSPC: MOS Prediction Based on Pairwise Comparison](https://mlsanity.com/paper/2306.10493/) (2023)
- [UrgentMOS: Unified Multi-Metric and Preference Learning for Robust Speech Quality Assessment](https://mlsanity.com/paper/2601.18438/) (2026)
- [Universal Preference-Score-based Pairwise Speech Quality Assessment](https://mlsanity.com/paper/2506.01455/) (2025)
- [Predicting pairwise preferences between TTS audio stimuli using parallel ratings data and anti-symmetric twin neural networks](https://mlsanity.com/paper/2209.11003/) (2022)

## Abstract (arXiv)

Mean opinion scores (MOS) are widely used for speech quality assessment, yet scalar labels are sensitive to rater variability and listening test differences. This introduces labeling noise, which limits the reliability of MOS prediction. Preference prediction reduces this variability as listeners compare signals directly, producing cleaner labels. We study MOS-free preference prediction and propose PrefSQA, which incorporates uncertainty-aware logits, an impairment attention head, and a module based on non-matching-reference comparisons. We use and refine five datasets, including MOS-derived and low-noise simulated sets with matching and non-matching content, experiment with human preference sets, and test on unseen data. Experiments show small improvements on MOS-derived data, while other sets reveal clear improvement over the baselines, highlighting the value of high-quality preference data and demonstrating the effectiveness of the proposed method.