mlsanityData to 14 Oct 2026

SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech

arXiv 2206.12132 ↗First posted 24 Jun 2022eess.AS cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we present SANE-TTS, a stable and natural end-to-end multilingual TTS model. By the difficulty of obtaining multilingual corpus for given speaker, training multilingual TTS model with monolingual corpora is unavoidable. We introduce speaker regularization loss that improves speech naturalness during cross-lingual synthesis as well as domain adversarial training, which is applied in other multilingual TTS models. Furthermore, by adding speaker regularization loss, replacing speaker embedding with zero vector in duration predictor stabilizes cross-lingual inference. With this replacement, our model generates speeches with moderate rhythm regardless of source speaker in cross-lingual synthesis. In MOS evaluation, SANE-TTS achieves naturalness score above 3.80 both in cross-lingual and intralingual synthesis, where the ground truth score is 3.99. Also, SANE-TTS maintains speaker similarity close to that of ground truth even in cross-lingual inference. Audio samples are available on our web page.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 44 of its 67 references that are in the map. First posted on arXiv 24 Jun 2022.

Builds on

44 of 67 references.

References · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  6. 2015Variational Inference with Normalizing Flowscited by 2,170
  7. 2018Self-Attention with Relative Position Representationscited by 1,120
  8. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  9. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  10. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  11. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  12. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
32 more
  1. 2018SDR - half-baked or well done?cited by 423
  2. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  3. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  4. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  5. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 216
  6. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 212
  7. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 191
  8. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  9. 2017Pyroomacoustics: A Python package for audio room simulations and array processing algorithmscited by 132
  10. 2020AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselinescited by 111
  11. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  12. 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 73
  13. 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
  14. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
  15. 2016Single-Channel Multi-Speaker Separation using Deep Clusteringcited by 45
  16. 2018Differentiable Consistency Constraints for Improved Deep Speech Enhancementcited by 35
  17. 2021Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modelingcited by 31
  18. 2020One Model, Many Languages: Meta-learning for Multilingual Text-to-Speechcited by 28
  19. 2019Conditioned-U-Net: Introducing a Control Mechanism in the U-Net for Multiple Source Separationscited by 27
  20. 2020Listen to What You Want: Neural Network-based Universal Sound Selectorcited by 27
  21. 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 22
  22. 2020Conditioned Source Separation for Music Instrument Performancescited by 18
  23. 2019Improving Universal Sound Separation Using Sound Classificationcited by 17
  24. 2021Compute and memory efficient universal sound source separationcited by 15
  25. 2018Class-conditional embeddings for music source separationcited by 12
  26. 2019Probabilistic Permutation Invariant Training for Speech Separationcited by 12
  27. 2021FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesiscited by 12
  28. 2019Cross-lingual Multi-speaker Text-to-speech Synthesis for Voice Cloning without Using Parallel Corpus for Unseen Speakerscited by 11
  29. 2020X-TaSNet: Robust and Accurate Time-Domain Speaker Extraction Networkcited by 9
  30. 2019Interrupted and cascaded permutation invariant training for speech separationcited by 8
  31. 2020Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-trainingcited by 6
  32. 2020Unified Gradient Reweighting for Model Biasing with Applications to Source Separationcited by 2

Built on by

Not cited yet.