mlsanityData to 7 Oct 2026

Deep Learning Based Assessment of Synthetic Speech Naturalness

arXiv 2104.11673 ↗First posted 23 Apr 2021cs.SD cs.AI cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a new objective prediction model for synthetic speech naturalness. It can be used to evaluate Text-To-Speech or Voice Conversion systems and works language independently. The model is trained end-to-end and based on a CNN-LSTM network that previously showed to give good results for speech quality estimation. We trained and tested the model on 16 different datasets, such as from the Blizzard Challenge and the Voice Conversion Challenge. Further, we show that the reliability of deep learning-based naturalness prediction can be improved by transfer learning from speech quality prediction models that are trained on objective POLQA scores. The proposed model is made publicly available and can, for example, be used to evaluate different TTS system configurations.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 3 of its 36 references that are in the map. First posted on arXiv 23 Apr 2021.

Builds on

3 of 36 references.

References · 3

  1. 2019MOSNet: Deep Learning based Objective Assessment for Voice Conversioncited by 126
  2. 2018The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methodscited by 68
  3. 2016AutoMOS: Learning a non-intrusive assessor of naturalness-of-speechcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 7; 2024: 7; 2025: 5; 2026: 4.

35 papers.

Sort:

By area:

  1. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  2. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
  3. 2023VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translationcited by 87
  4. 2022The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generationcited by 40
  5. 2023Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022cited by 29
  6. 2021LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speechcited by 24
  7. 2024Building a Taiwanese Mandarin Spoken Language Model: A First Attemptcited by 22
  8. 2023Boosting Large Language Model for Speech Synthesis: An Empirical Studycited by 21
  9. 2022SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesiscited by 18
  10. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  11. 2023EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speechcited by 14
  12. 2024Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gapcited by 11
23 more
  1. 2025Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Studycited by 11
  2. 2024TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservationcited by 10
  3. 20241000 African Voices: Advancing inclusive multi-speaker multi-accent speech synthesiscited by 8
  4. 2022Robust One-Shot Singing Voice Conversioncited by 5
  5. 2021SVSNet: An End-to-end Speaker Voice Similarity Assessment Modelcited by 4
  6. 2021Facetron: A Multi-speaker Face-to-Speech Model based on Cross-modal Latent Representationscited by 4
  7. 2025Pairwise Evaluation of Accent Similarity in Speech Synthesiscited by 4
  8. 2024On the Problem of Text-To-Speech Model Selection for Synthetic Data Generation in Automatic Speech Recognitioncited by 3
  9. 2022Learning and controlling the source-filter representation of speech with a variational autoencodercited by 2
  10. 2022Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Featurescited by 2
  11. 2025APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speechcited by 2
  12. 2022Wideband Audio Waveform Evaluation Networks: Efficient, Accurate Estimation of Speech Qualitiescited by 1
  13. 2022Predicting pairwise preferences between TTS audio stimuli using parallel ratings data and anti-symmetric twin neural networkscited by 1
  14. 2023Cross-modal Face- and Voice-style Transfercited by 1
  15. 2023Iteratively Improving Speech Recognition and Voice Conversioncited by 1
  16. 2024Humane Speech Synthesis through Zero-Shot Emotion and Disfluency Generationcited by 1
  17. 2025VoiceCloak: A Multi-Dimensional Defense Framework against Unauthorized Diffusion-based Voice Cloningcited by 1
  18. 2026The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speechcited by 1
  19. 2022RedPen: Region- and Reason-Annotated Dataset of Unnatural Speechcited by 0
  20. 2025Balalaika: Data-Centric, Prosody-Aware Annotation Pipeline for Russian Speechcited by 0
  21. 2026LuxEmo: Expressive Text-to-Speech Corpus for Luxembourgishcited by 0
  22. 2026LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answeringcited by 0
  23. 2026Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architecturescited by 0