# Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditions

arXiv: https://arxiv.org/abs/2509.24457 · HTML: https://mlsanity.com/paper/2509.24457/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Builds on (most cited first, 30 of 34)

- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Moshi: a speech-text foundation model for real-time dialogue](https://mlsanity.com/paper/2410.00037/) (2024)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech](https://mlsanity.com/paper/2409.09305/) (2024)
- [ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit](https://mlsanity.com/paper/1910.10909/) (2019)
- [TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudio](https://mlsanity.com/paper/2304.01448/) (2023)
- [The VoiceMOS Challenge 2022](https://mlsanity.com/paper/2203.11389/) (2022)
- [HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement](https://mlsanity.com/paper/2203.13086/) (2022)
- [Scaling Transformers for Low-Bitrate High-Quality Speech Coding](https://mlsanity.com/paper/2411.19842/) (2024)
- [AudioDec: An Open-source Streaming High-fidelity Neural Audio Codec](https://mlsanity.com/paper/2305.16608/) (2023)
- [BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec](https://mlsanity.com/paper/2409.05377/) (2024)
- [SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Sound](https://mlsanity.com/paper/2405.00233/) (2024)
- [SNAC: Multi-Scale Neural Audio Codec](https://mlsanity.com/paper/2410.14411/) (2024)
- [SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metrics](https://mlsanity.com/paper/2401.16812/) (2024)
- [VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Music](https://mlsanity.com/paper/2412.17667/) (2024)
- [An Open source Implementation of ITU-T Recommendation P.808 with Validation](https://mlsanity.com/paper/2005.08138/) (2020)
- [SCOREQ: Speech Quality Assessment with Contrastive Regression](https://mlsanity.com/paper/2410.06675/) (2024)
- [ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models](https://mlsanity.com/paper/2401.17230/) (2024)
- [DualCodec: A Low-Frame-Rate, Semantically-Enhanced Neural Audio Codec for Speech Generation](https://mlsanity.com/paper/2505.13000/) (2025)
- [The VoiceMOS Challenge 2024: Beyond Speech Quality Prediction](https://mlsanity.com/paper/2409.07001/) (2024)
- [Objective Measures of Perceptual Audio Quality Reviewed: An Evaluation of Their Application Domain Dependence](https://mlsanity.com/paper/2110.11438/) (2021)
- [NORESQA: A Framework for Speech Quality Assessment using Non-Matching References](https://mlsanity.com/paper/2109.08125/) (2021)
- [WARP-Q: Quality Prediction For Generative Neural Speech Codecs](https://mlsanity.com/paper/2102.10449/) (2021)
- [The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement](https://mlsanity.com/paper/2406.03460/) (2024)
- [MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Models](https://mlsanity.com/paper/2411.03715/) (2024)
- [ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding](https://mlsanity.com/paper/2207.09514/) (2022)
- [The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domains](https://mlsanity.com/paper/2310.02640/) (2023)
- [PLCMOS -- a data-driven non-intrusive metric for the evaluation of packet loss concealment algorithms](https://mlsanity.com/paper/2305.15127/) (2023)

## Abstract (arXiv)

Objective speech-quality metrics are widely used to assess codec performance. However, for neural codecs, it is often unclear which metrics provide reliable quality estimates. To address this, we evaluated 45 objective metrics by correlating their scores with subjective listening scores for clean speech across 17 codec conditions. Neural-based metrics such as scoreq and utmos achieved the highest Pearson correlations with subjective scores. Further analysis across different subjective quality ranges revealed that non-intrusive metrics tend to saturate at high subjective quality levels.