# SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis

arXiv: https://arxiv.org/abs/2204.03040 · HTML: https://mlsanity.com/paper/2204.03040/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 22 of 22)

- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [Generalization Ability of MOS Prediction Networks](https://mlsanity.com/paper/2110.02635/) (2021)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling](https://mlsanity.com/paper/2010.04301/) (2020)
- [The VoiceMOS Challenge 2022](https://mlsanity.com/paper/2203.11389/) (2022)
- [Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM](https://mlsanity.com/paper/1808.05344/) (2018)
- [MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network](https://mlsanity.com/paper/2103.00110/) (2021)
- [Deep Learning Based Assessment of Synthetic Speech Naturalness](https://mlsanity.com/paper/2104.11673/) (2021)
- [LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech](https://mlsanity.com/paper/2110.09103/) (2021)
- [AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech](https://mlsanity.com/paper/1611.09207/) (2016)
- [How do Voices from Past Speech Synthesis Challenges Compare Today?](https://mlsanity.com/paper/2105.02373/) (2021)
- [Controllable neural text-to-speech synthesis using intuitive prosodic features](https://mlsanity.com/paper/2009.06775/) (2020)
- [High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency](https://mlsanity.com/paper/2111.09052/) (2021)
- [Bunched LPCNet : Vocoder for Low-cost Neural Text-To-Speech Systems](https://mlsanity.com/paper/2008.04574/) (2020)
- [Formal Definition of QoE Metrics](https://mlsanity.com/paper/1607.00321/) (2016)
- [Comparison of Speech Representations for Automatic Quality Estimation in Multi-Speaker Text-to-Speech Synthesis](https://mlsanity.com/paper/2002.12645/) (2020)
- [Word-Level Style Control for Expressive, Non-attentive Speech Synthesis](https://mlsanity.com/paper/2111.10173/) (2021)
- [Deep MOS Predictor for Synthetic Speech Using Cluster-Based Modeling](https://mlsanity.com/paper/2008.03710/) (2020)

## Built on by (most cited first, 18 of 18)

- [The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech](https://mlsanity.com/paper/2409.09305/) (2024)
- [Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation](https://mlsanity.com/paper/2409.16644/) (2024)
- [TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems](https://mlsanity.com/paper/2506.19441/) (2025)
- [Flex-Judge: Text-Only Reasoning Unleashes Zero-Shot Multimodal Evaluators](https://mlsanity.com/paper/2505.18601/) (2025)
- [Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciation](https://mlsanity.com/paper/2210.17264/) (2022)
- [Resource-Efficient Fine-Tuning Strategies for Automatic MOS Prediction in Text-to-Speech for Low-Resource Languages](https://mlsanity.com/paper/2305.19396/) (2023)
- [Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Features](https://mlsanity.com/paper/2211.00342/) (2022)
- [Strategies in Transfer Learning for Low-Resource Speech Synthesis: Phone Mapping, Features Input, and Source Language Selection](https://mlsanity.com/paper/2306.12040/) (2023)
- [Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokens](https://mlsanity.com/paper/2509.20485/) (2025)
- [TTS-PRISM: A Perceptual Reasoning and Interpretable Speech Model for Fine-Grained Diagnosis](https://mlsanity.com/paper/2604.22225/) (2026)
- [The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speech](https://mlsanity.com/paper/2609.13150/) (2026)
- [EventTrojan: Manipulating Non-Intrusive Speech Quality Assessment via Imperceptible Events](https://mlsanity.com/paper/2309.01480/) (2023)
- [It HAS to be Subjective: Human Annotator Simulation via Zero-shot Density Estimation](https://mlsanity.com/paper/2310.00486/) (2023)
- [Generating Multilingual Gender-Ambiguous Text-to-Speech Voices](https://mlsanity.com/paper/2211.00375/) (2022)
- [The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech](https://mlsanity.com/paper/2306.00535/) (2023)
- [SALF-MOS: Speaker Agnostic Latent Features Downsampled for MOS Prediction](https://mlsanity.com/paper/2506.02082/) (2025)
- [PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets](https://mlsanity.com/paper/2606.19597/) (2026)
- [DAMOS: Learning Distortion-Aware Speech Quality Assessment through Explicit Distortion Localization](https://mlsanity.com/paper/2608.21176/) (2026)

## Abstract (arXiv)

In this work, we present the SOMOS dataset, the first large-scale mean opinion scores (MOS) dataset consisting of solely neural text-to-speech (TTS) samples. It can be employed to train automatic MOS prediction systems focused on the assessment of modern synthesizers, and can stimulate advancements in acoustic model evaluation. It consists of 20K synthetic utterances of the LJ Speech voice, a public domain speech dataset which is a common benchmark for building neural acoustic models and vocoders. Utterances are generated from 200 TTS systems including vanilla neural acoustic models as well as models which allow prosodic variations. An LPCNet vocoder is used for all systems, so that the samples' variation depends only on the acoustic models. The synthesized utterances provide balanced and adequate domain and length coverage. We collect MOS naturalness evaluations on 3 English Amazon Mechanical Turk locales and share practices leading to reliable crowdsourced annotations for this task. We provide baseline results of state-of-the-art MOS prediction models on the SOMOS dataset and show the limitations that such models face when assigned to evaluate TTS utterances.