mlsanityData to 7 Oct 2026

SQuId: Measuring Speech Naturalness in Many Languages

arXiv 2210.06324 ↗First posted 12 Oct 2022cs.CL cs.LG cs.SDcited by 10

Not on the map. Why?

Abstract (arXiv)

Much of text-to-speech research relies on human evaluation, which incurs heavy costs and slows down the development process. The problem is particularly acute in heavily multilingual applications, where recruiting and polling judges can take weeks. We introduce SQuId (Speech Quality Identification), a multilingual naturalness prediction model trained on over a million ratings and tested in 65 locales-the largest effort of this type to date. The main insight is that training one model on many locales consistently outperforms mono-locale baselines. We present our task, the model, and show that it outperforms a competitive baseline based on w2v-BERT and VoiceMOS by 50.0%. We then demonstrate the effectiveness of cross-locale transfer during fine-tuning and highlight its effect on zero-shot locales, i.e., locales for which there is no fine-tuning data. Through a series of analyses, we highlight the role of non-linguistic effects such as sound artifacts in cross-locale transfer. Finally, we present the effect of our design decision, e.g., model size, pre-training diversity, and language rebalancing with several ablation experiments.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 34 of its 47 references that are in the map. First posted on arXiv 12 Oct 2022.

Builds on

34 of 47 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  4. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  5. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  6. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  7. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  8. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  9. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  10. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  11. 2019Massively Multilingual Neural Machine Translationcited by 246
  12. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
22 more
  1. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  2. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 148
  3. 2019MOSNet: Deep Learning based Objective Assessment for Voice Conversioncited by 126
  4. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  5. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 92
  6. 2020Universal Phone Recognition with a Multilingual Allophone Systemcited by 81
  7. 2021Generalization Ability of MOS Prediction Networkscited by 69
  8. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
  9. 2022The VoiceMOS Challenge 2022cited by 57
  10. 2017Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Modelcited by 37
  11. 2021MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Networkcited by 37
  12. 2021Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Featurescited by 32
  13. 2021LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speechcited by 24
  14. 2016AutoMOS: Learning a non-intrusive assessor of naturalness-of-speechcited by 22
  15. 2021Utilizing Self-supervised Representations for MOS Predictioncited by 20
  16. 2021Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesiscited by 13
  17. 2020That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languagescited by 10
  18. 2020How Phonotactics Affect Multilingual and Zero-shot ASR Performancecited by 10
  19. 2020Neural MOS Prediction for Synthesized Speech Using Multi-Task Learning With Spoofing Detection and Spoofing Type Classificationcited by 7
  20. 2020Comparison of Speech Representations for Automatic Quality Estimation in Multi-Speaker Text-to-Speech Synthesiscited by 5
  21. 2022A Comparison of Deep Learning MOS Predictors for Speech Synthesis Qualitycited by 5
  22. 2022Unify and Conquer: How Phonetic Feature Representation Affects Polyglot Text-To-Speech (TTS)cited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 5; 2024: 1; 2025: 2.

10 papers.

  1. 2023E3 TTS: Easy End-to-End Diffusion-based Text to Speechcited by 30
  2. 2025EmergentTTS-Eval: Evaluating TTS Models on Complex Prosodic, Expressiveness, and Linguistic Challenges Using Model-as-a-Judgecited by 24
  3. 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
  4. 2022SpeechLMScore: Evaluating speech generation using speech language modelcited by 19
  5. 2022Virtuoso: Massive Multilingual Speech-Text Joint Semi-Supervised Learning for Text-To-Speechcited by 15
  6. 2024FLEURS-R: A Restored Multilingual Speech Corpus for Generation Taskscited by 13
  7. 2023Translatotron 3: Speech to Speech Translation with Monolingual Datacited by 12
  8. 2023Evaluating Speech Synthesis by Training Recognizers on Synthetic Speechcited by 3
  9. 2025Miipher-2: A Universal Speech Restoration Model for Million-Hour Scale Data Restorationcited by 1
  10. 2023Reduce, Reuse, Recycle: Is Perturbed Data better than Other Language augmentation for Low Resource Self-Supervised Speech Modelscited by 0