mlsanityData to 14 Oct 2026

Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditions

arXiv 2509.24457 ↗First posted 29 Sep 2025eess.AS cs.AI cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Objective speech-quality metrics are widely used to assess codec performance. However, for neural codecs, it is often unclear which metrics provide reliable quality estimates. To address this, we evaluated 45 objective metrics by correlating their scores with subjective listening scores for clean speech across 17 codec conditions. Neural-based metrics such as scoreq and utmos achieved the highest Pearson correlations with subjective scores. Further analysis across different subjective quality ranges revealed that non-intrusive metrics tend to saturate at high subjective quality levels.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 29 Sep 2025.

Builds on

34 of 50 references.

References · 34

  1. 2022High Fidelity Neural Audio Compressioncited by 687
  2. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  3. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  4. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  5. 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 192
  6. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
  7. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
  8. 2023TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudiocited by 59
  9. 2022The VoiceMOS Challenge 2022cited by 57
  10. 2022HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancementcited by 49
  11. 2024Scaling Transformers for Low-Bitrate High-Quality Speech Codingcited by 49
  12. 2023AudioDec: An Open-source Streaming High-fidelity Neural Audio Codeccited by 47
22 more
  1. 2024BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codeccited by 46
  2. 2024SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Soundcited by 45
  3. 2024SNAC: Multi-Scale Neural Audio Codeccited by 43
  4. 2024SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metricscited by 38
  5. 2024VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Musiccited by 30
  6. 2020An Open source Implementation of ITU-T Recommendation P.808 with Validationcited by 27
  7. 2024SCOREQ: Speech Quality Assessment with Contrastive Regressioncited by 24
  8. 2024ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf modelscited by 22
  9. 2025DualCodec: A Low-Frame-Rate, Semantically-Enhanced Neural Audio Codec for Speech Generationcited by 21
  10. 2024The VoiceMOS Challenge 2024: Beyond Speech Quality Predictioncited by 17
  11. 2021Objective Measures of Perceptual Audio Quality Reviewed: An Evaluation of Their Application Domain Dependencecited by 14
  12. 2021NORESQA: A Framework for Speech Quality Assessment using Non-Matching Referencescited by 13
  13. 2021WARP-Q: Quality Prediction For Generative Neural Speech Codecscited by 12
  14. 2024The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancementcited by 12
  15. 2024MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Modelscited by 12
  16. 2022ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understandingcited by 8
  17. 2023The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domainscited by 8
  18. 2023PLCMOS -- a data-driven non-intrusive metric for the evaluation of packet loss concealment algorithmscited by 7
  19. 2023Multi-dimensional Speech Quality Assessment in Crowdsourcingcited by 4
  20. 2024ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized Transformerscited by 3
  21. 2025Crowdsourcing MUSHRA Tests in the Age of Generative Speech Technologies: A Comparative Analysis of Subjective and Objective Testing Methodscited by 1
  22. 2025MUSHRA-1S: A scalable and sensitive test approach for evaluating top-tier speech processing systemscited by 1

Built on by

Not cited yet.