mlsanityData to 7 Oct 2026

SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis

arXiv 2204.03040 ↗First posted 6 Apr 2022cs.SD cs.CL cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

In this work, we present the SOMOS dataset, the first large-scale mean opinion scores (MOS) dataset consisting of solely neural text-to-speech (TTS) samples. It can be employed to train automatic MOS prediction systems focused on the assessment of modern synthesizers, and can stimulate advancements in acoustic model evaluation. It consists of 20K synthetic utterances of the LJ Speech voice, a public domain speech dataset which is a common benchmark for building neural acoustic models and vocoders. Utterances are generated from 200 TTS systems including vanilla neural acoustic models as well as models which allow prosodic variations. An LPCNet vocoder is used for all systems, so that the samples' variation depends only on the acoustic models. The synthesized utterances provide balanced and adequate domain and length coverage. We collect MOS naturalness evaluations on 3 English Amazon Mechanical Turk locales and share practices leading to reliable crowdsourced annotations for this task. We provide baseline results of state-of-the-art MOS prediction models on the SOMOS dataset and show the limitations that such models face when assigned to evaluate TTS utterances.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 22 of its 42 references that are in the map. First posted on arXiv 6 Apr 2022.

Builds on

22 of 42 references.

References · 22

  1. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  2. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  3. 2019ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detectioncited by 173
  4. 2019MOSNet: Deep Learning based Objective Assessment for Voice Conversioncited by 126
  5. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
  6. 2021Generalization Ability of MOS Prediction Networkscited by 69
  7. 2018The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methodscited by 68
  8. 2020Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modelingcited by 65
  9. 2022The VoiceMOS Challenge 2022cited by 57
  10. 2021MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Networkcited by 37
  11. 2018Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTMcited by 36
  12. 2021Deep Learning Based Assessment of Synthetic Speech Naturalnesscited by 35
10 more
  1. 2021LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speechcited by 24
  2. 2016AutoMOS: Learning a non-intrusive assessor of naturalness-of-speechcited by 22
  3. 2021How do Voices from Past Speech Synthesis Challenges Compare Today?cited by 21
  4. 2020Controllable neural text-to-speech synthesis using intuitive prosodic featurescited by 19
  5. 2021High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latencycited by 19
  6. 2020Bunched LPCNet : Vocoder for Low-cost Neural Text-To-Speech Systemscited by 16
  7. 2016Formal Definition of QoE Metricscited by 7
  8. 2020Comparison of Speech Representations for Automatic Quality Estimation in Multi-Speaker Text-to-Speech Synthesiscited by 5
  9. 2021Word-Level Style Control for Expressive, Non-attentive Speech Synthesiscited by 5
  10. 2020Deep MOS Predictor for Synthetic Speech Using Cluster-Based Modelingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 2; 2025: 4; 2026: 4.

18 papers.

Sort:

By area:

  1. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 78
  2. 2024Enabling Auditory Large Language Models for Automatic Speech Quality Evaluationcited by 13
  3. 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 7
  4. 2025Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluatorscited by 6
  5. 2022Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciationcited by 3
  6. 2023Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languagescited by 3
  7. 2022Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Featurescited by 2
  8. 2023Strategies in Transfer Learning for Low-Resource Speech Synthesis: Phone Mapping, Features Input, and Source Language Selectioncited by 2
  9. 2025Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokenscited by 2
  10. 2026TTS-PRISM: A Perceptual Reasoning and Interpretable Speech Model for Fine-Grained Diagnosiscited by 2
  11. 2023EventTrojan: Manipulating Non-Intrusive Speech Quality Assessment via Imperceptible Eventscited by 1
  12. 2023It HAS to be Subjective: Human Annotator Simulation via Zero-shot Density Estimationcited by 1
6 more
  1. 2026The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speechcited by 1
  2. 2022Generating Multilingual Gender-Ambiguous Text-to-Speech Voicescited by 0
  3. 2023The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speechcited by 0
  4. 2025SALF-MOS: Speaker Agnostic Latent Features Downsampled for MOS Predictioncited by 0
  5. 2026PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasetscited by 0
  6. 2026DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localizationcited by 0