Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations
Not on the map. Why?
Abstract (arXiv)
Mean opinion score (MOS) prediction models are widely used as proxy metrics in text-to-speech (TTS) research, yet their ability to capture quality differences beyond acoustic fidelity remains unclear. We investigate this via controlled perturbations on speech: acoustic degradation, prosodic errors, and manipulation of speaker-specific characteristics such as pitch and speaking rate. We obtained MOS predictions for these speech samples from both human listeners and the model, and analyzed the differences in their perceptual characteristics. Results show that most models track acoustic degradation well, while all are insensitive to prosodic errors despite large subjective score drops. For speaker characteristics, models exhibit a double dissociation: strong mean fundamental frequency (F0) biases absent in human ratings, yet insensitivity to speaking rate and F0 variability that humans notice. These findings highlight limitations of scalar MOS prediction beyond acoustic fidelity.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 18 Jun 2026.
Builds on
26 of 35 references.
References · 26
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
- 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
- 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
- 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 192
- 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
14 more
- 2021Generalization Ability of MOS Prediction Networkscited by 69
- 2022NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesiscited by 38
- 2024VERSA: A Versatile Evaluation Toolkit for Speech, Audio, and Musiccited by 30
- 2024ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speechcited by 26
- 2024The VoiceMOS Challenge 2024: Beyond Speech Quality Predictioncited by 17
- 2024MOS-Bench: Benchmarking Generalization Abilities of Subjective Speech Quality Assessment Modelscited by 12
- 2022Source-Filter HiFi-GAN: Fast and Pitch Controllable High-Fidelity Neural Vocodercited by 11
- 2024SingMOS: An extensive Open-Source Singing Voice Dataset for MOS Predictioncited by 9
- 2025Measuring Prosody Diversity in Zero-Shot TTS: A New Metric, Benchmark, and Explorationcited by 7
- 2025SHEET: A Multi-purpose Open-source Speech Human Evaluation Estimation Toolkitcited by 4
- 2024Pitch-and-Spectrum-Aware Singing Quality Assessment with Bias Correction and Model Fusioncited by 3
- 2025Uni-VERSA: Versatile Speech Assessment with a Unified Networkcited by 3
- 2022Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Featurescited by 2
- 2024Analytic Study of Text-Free Speech Synthesis for Raw Audio using a Self-Supervised Learning Modelcited by 1
Built on by
Not cited yet.