# AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech

arXiv: https://arxiv.org/abs/1611.09207 · HTML: https://mlsanity.com/paper/1611.09207/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2016.

## Built on by (most cited first, 22 of 22)

- [CDPAM: Contrastive learning for perceptual audio similarity](https://mlsanity.com/paper/2102.05109/) (2021)
- [Deep Learning Based Assessment of Synthetic Speech Naturalness](https://mlsanity.com/paper/2104.11673/) (2021)
- [Utilizing Self-supervised Representations for MOS Prediction](https://mlsanity.com/paper/2104.03017/) (2021)
- [SpeechLMScore: Evaluating speech generation using speech language model](https://mlsanity.com/paper/2212.04559/) (2022)
- [SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis](https://mlsanity.com/paper/2204.03040/) (2022)
- [Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation](https://mlsanity.com/paper/2409.16644/) (2024)
- [Translatotron 3: Speech to Speech Translation with Monolingual Data](https://mlsanity.com/paper/2305.17547/) (2023)
- [SQuId: Measuring Speech Naturalness in Many Languages](https://mlsanity.com/paper/2210.06324/) (2022)
- [On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognition](https://mlsanity.com/paper/2310.08132/) (2023)
- [Automatic Evaluation of Speaker Similarity](https://mlsanity.com/paper/2207.00344/) (2022)
- [Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Features](https://mlsanity.com/paper/2211.00342/) (2022)
- [Evaluation of Speech Representations for MOS prediction](https://mlsanity.com/paper/2306.09979/) (2023)
- [APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speech](https://mlsanity.com/paper/2504.20447/) (2025)
- [Predicting pairwise preferences between TTS audio stimuli using parallel ratings data and anti-symmetric twin neural networks](https://mlsanity.com/paper/2209.11003/) (2022)
- [EventTrojan: Manipulating Non-Intrusive Speech Quality Assessment via Imperceptible Events](https://mlsanity.com/paper/2309.01480/) (2023)
- [It HAS to be Subjective: Human Annotator Simulation via Zero-shot Density Estimation](https://mlsanity.com/paper/2310.00486/) (2023)
- [CORN: Co-Trained Full- And No-Reference Speech Quality Assessment](https://mlsanity.com/paper/2310.09388/) (2023)
- [CCATMos: Convolutional Context-aware Transformer Network for Non-intrusive Speech Quality Assessment](https://mlsanity.com/paper/2211.02577/) (2022)
- [Personalized Audio Quality Preference Prediction](https://mlsanity.com/paper/2302.08130/) (2023)
- [SALF-MOS: Speaker Agnostic Latent Features Downsampled for MOS Prediction](https://mlsanity.com/paper/2506.02082/) (2025)
- [PASQA: Pitch-Accent-Focused Speech Quality Assessment Model Trained on Synthetic Speech with Accent Errors](https://mlsanity.com/paper/2606.20137/) (2026)
- [Beyond Naturalness: Probing Automated Text-To-Speech Evaluators on Linguistically Grounded Dimensions](https://mlsanity.com/paper/2608.09930/) (2026)

## Abstract (arXiv)

Developers of text-to-speech synthesizers (TTS) often make use of human raters to assess the quality of synthesized speech. We demonstrate that we can model human raters' mean opinion scores (MOS) of synthesized speech using a deep recurrent neural network whose inputs consist solely of a raw waveform. Our best models provide utterance-level estimates of MOS only moderately inferior to sampled human ratings, as shown by Pearson and Spearman correlations. When multiple utterances are scored and averaged, a scenario common in synthesizer quality assessment, AutoMOS achieves correlations approaching those of human raters. The AutoMOS model has a number of applications, such as the ability to explore the parameter space of a speech synthesizer without requiring a human-in-the-loop.