mlsanityData to 7 Oct 2026

Generalization Ability of MOS Prediction Networks

arXiv 2110.02635 ↗First posted 6 Oct 2021eess.AScited by 69

Not on the map. Why?

Abstract (arXiv)

Automatic methods to predict listener opinions of synthesized speech remain elusive since listeners, systems being evaluated, characteristics of the speech, and even the instructions given and the rating scale all vary from test to test. While automatic predictors for metrics such as mean opinion score (MOS) can achieve high prediction accuracy on samples from the same test, they typically fail to generalize well to new listening test contexts. In this paper, using a variety of networks for MOS prediction including MOSNet and self-supervised speech models such as wav2vec2, we investigate their performance on data from different listening tests in both zero-shot and fine-tuned settings. We find that wav2vec2 models fine-tuned for MOS prediction have good generalization capability to out-of-domain data even for the most challenging case of utterance-level predictions in the zero-shot setting, and that fine-tuning to in-domain data can improve predictions. We also observe that unseen systems are especially challenging for MOS prediction models.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Oct 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 12; 2023: 17; 2024: 10; 2025: 9; 2026: 18.

69 papers.

Sort:

By area:

  1. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 78
  2. 2023Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation taskscited by 71
  3. 2021Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Featurescited by 32
  4. 2023The Singing Voice Conversion Challenge 2023cited by 25
  5. 2025Audio Large Language Models Can Be Descriptive Speech Quality Evaluatorscited by 25
  6. 2021LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speechcited by 24
  7. 2024Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedbackcited by 22
  8. 2022SpeechLMScore: Evaluating speech generation using speech language modelcited by 19
  9. 2024Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedbackcited by 19
  10. 2022SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesiscited by 18
  11. 2024Enabling Auditory Large Language Models for Automatic Speech Quality Evaluationcited by 13
  12. 2022Fusion of Self-supervised Learned Models for MOS Predictioncited by 10
38 more
  1. 2022SQuId: Measuring Speech Naturalness in Many Languagescited by 10
  2. 2024Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimizationcited by 10
  3. 2025FELLE: Autoregressive Speech Synthesis with Token-Wise Coarse-to-Fine Flow Matchingcited by 8
  4. 2021InQSS: a speech intelligibility and quality assessment model using a multi-task learning networkcited by 7
  5. 2022DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scorescited by 7
  6. 2023MOSPC: MOS Prediction Based on Pairwise Comparisoncited by 7
  7. 2024A Benchmark for Multi-speaker Anonymizationcited by 7
  8. 2022MTI-Net: A Multi-Target Speech Intelligibility Prediction Modelcited by 6
  9. 2024Malacopula: adversarial automatic speaker verification attacks using a neural-based generalised Hammerstein modelcited by 6
  10. 2026SongBench: A Fine-Grained Multi-Aspect Benchmark for Song Quality Assessmentcited by 6
  11. 2023On the Use of Self-Supervised Speech Representations in Spontaneous Speech Synthesiscited by 4
  12. 2023Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadatacited by 4
  13. 2025Robust One-step Speech Enhancement via Consistency Distillationcited by 4
  14. 2023A Comparative Study of Self-Supervised Speech Representations in Read and Spontaneous TTScited by 3
  15. 2023Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languagescited by 3
  16. 2023Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals using Self Supervised Speech Representationscited by 3
  17. 2023Evaluating Speech Synthesis by Training Recognizers on Synthetic Speechcited by 3
  18. 2024SpeechComposer: Unifying Multiple Speech Tasks with Prompt Compositioncited by 3
  19. 2025QAMRO: Quality-aware Adaptive Margin Ranking Optimization for Human-aligned Assessment of Audio Generation Systemscited by 3
  20. 2022Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Featurescited by 2
  21. 2023Evaluation of Speech Representations for MOS predictioncited by 2
  22. 2023Strategies in Transfer Learning for Low-Resource Speech Synthesis: Phone Mapping, Features Input, and Source Language Selectioncited by 2
  23. 2024A Collection of Pragmatic-Similarity Judgments over Spoken Dialog Utterancescited by 2
  24. 2025APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speechcited by 2
  25. 2025DRASP: A Dual-Resolution Attentive Statistics Pooling Framework for Automatic MOS Predictioncited by 2
  26. 2026Reflecting Twice before Speaking with Empathy: Self-Reflective Alternating Inference for Empathy-Aware End-to-End Spoken Dialoguecited by 2
  27. 2022Wideband Audio Waveform Evaluation Networks: Efficient, Accurate Estimation of Speech Qualitiescited by 1
  28. 2022Using Rater and System Metadata to Explain Variance in the VoiceMOS Challenge 2022 Datasetcited by 1
  29. 2023Multi-Task Pseudo-Label Learning for Non-Intrusive Speech Quality Assessment Modelcited by 1
  30. 2023EventTrojan: Manipulating Non-Intrusive Speech Quality Assessment via Imperceptible Eventscited by 1
  31. 2024SVSNet+: Enhancing Speaker Voice Similarity Assessment Models with Representations from Speech Foundation Modelscited by 1
  32. 2025JSQA: Speech Quality Assessment with Perceptually-Inspired Contrastive Pretraining Based on JND Audio Pairscited by 1
  33. 2025Improving Perceptual Audio Aesthetic Assessment via Triplet Loss and Self-Supervised Embeddingscited by 1
  34. 2026AnimeScore: A Preference-Based Dataset and Framework for Evaluating Anime-Like Speech Stylecited by 1
  35. 2026The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speechcited by 1
  36. 2022Improving Self-Supervised Learning-based MOS Prediction Networkscited by 0
  37. 2022Comparison of Speech Representations for the MOS Prediction Systemcited by 0
  38. 2022RedPen: Region- and Reason-Annotated Dataset of Unnatural Speechcited by 0

50 most cited of 69 All 69 (JSON)