# WhiSQA: Non-Intrusive Speech Quality Prediction Using Whisper Encoder Features

arXiv: https://arxiv.org/abs/2508.02210 · HTML: https://mlsanity.com/paper/2508.02210/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors](https://mlsanity.com/paper/2110.01763/) (2021)
- [Comparative layer-wise analysis of self-supervised speech models](https://mlsanity.com/paper/2211.03929/) (2022)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudio](https://mlsanity.com/paper/2304.01448/) (2023)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [Voices Obscured in Complex Environmental Settings (VOICES) corpus](https://mlsanity.com/paper/1804.05053/) (2018)
- [MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech](https://mlsanity.com/paper/2110.05866/) (2021)
- [ConferencingSpeech 2022 Challenge: Non-intrusive Objective Speech Quality Assessment (NISQA) Challenge for Online Conferencing Applications](https://mlsanity.com/paper/2203.16032/) (2022)
- [The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement](https://mlsanity.com/paper/2406.03460/) (2024)
- [Non-Intrusive Speech Intelligibility Prediction for Hearing-Impaired Users using Intermediate ASR Features and Human Memory Models](https://mlsanity.com/paper/2401.13611/) (2024)
- [DNN No-Reference PSTN Speech Quality Prediction](https://mlsanity.com/paper/2007.14598/) (2020)
- [Perceive and predict: self-supervised speech representation based loss functions for speech enhancement](https://mlsanity.com/paper/2301.04388/) (2023)
- [A Pyramid Recurrent Network for Predicting Crowdsourced Speech-Quality Ratings of Real-World Signals](https://mlsanity.com/paper/2007.15797/) (2020)
- [MetricGAN+/-: Increasing Robustness of Noise Reduction on Unseen Data](https://mlsanity.com/paper/2203.12369/) (2022)
- [The CHiME-7 UDASE task: Unsupervised domain adaptation for conversational speech enhancement](https://mlsanity.com/paper/2307.03533/) (2023)
- [Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals using Self Supervised Speech Representations](https://mlsanity.com/paper/2307.13423/) (2023)
- [Analysis of XLS-R for Speech Quality Assessment](https://mlsanity.com/paper/2308.12077/) (2023)
- [Multi-CMGAN+/+: Leveraging Multi-Objective Speech Quality Metric Prediction for Speech Enhancement](https://mlsanity.com/paper/2312.08979/) (2023)

## Built on by (most cited first, 2 of 2)

- [Flowing Straighter with Conditional Flow Matching for Accurate Speech Enhancement](https://mlsanity.com/paper/2508.20584/) (2025)
- [A Comprehensive Objective Evaluation of Modern Text-to-Speech for Turkish Using Speech Quality Assessment Models](https://mlsanity.com/paper/2610.06057/) (2026)

## Abstract (arXiv)

There has been significant research effort developing neural-network-based predictors of SQ in recent years. While a primary objective has been to develop non-intrusive, i.e.~reference-free, metrics to assess the performance of SE systems, recent work has also investigated the direct inference of neural SQ predictors within the loss function of downstream speech tasks. To aid in the training of SQ predictors, several large datasets of audio with corresponding human labels of quality have been created. Recent work in this area has shown that speech representations derived from large unsupervised or semi-supervised foundational speech models are useful input feature representations for neural SQ prediction. In this work, a novel and robust SQ predictor is proposed based on feature representations extracted from an ASR model, found to be a powerful input feature for the SQ prediction task. The proposed system achieves higher correlation with human MOS ratings than recent approaches on all NISQA test sets and shows significantly better domain adaption compared to the commonly used DNSMOS metric.