mlsanityData to 7 Oct 2026

Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciation

arXiv 2210.17264 ↗First posted 31 Oct 2022cs.SD cs.CL cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper presents a method for end-to-end cross-lingual text-to-speech (TTS) which aims to preserve the target language's pronunciation regardless of the original speaker's language. The model used is based on a non-attentive Tacotron architecture, where the decoder has been replaced with a normalizing flow network conditioned on the speaker identity, allowing both TTS and voice conversion (VC) to be performed by the same model due to the inherent linguistic content and speaker identity disentanglement. When used in a cross-lingual setting, acoustic features are initially produced with a native speaker of the target language and then voice conversion is applied by the same model in order to convert these features to the target speaker's voice. We verify through objective and subjective evaluations that our method can have benefits compared to baseline cross-lingual synthesis. By including speakers averaging 7.5 minutes of speech, we also present positive results on low-resource scenarios.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

20 of 32 references.

References · 20

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  3. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  4. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  5. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  6. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  7. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  8. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  9. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  10. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
  11. 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
  12. 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
8 more
  1. 2020Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modelingcited by 65
  2. 2020One Model, Many Languages: Meta-learning for Multilingual Text-to-Speechcited by 28
  3. 2021S3PRL-VC: Open-source Voice Conversion Framework with Self-supervised Speech Representationscited by 22
  4. 2021High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latencycited by 19
  5. 2022SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesiscited by 18
  6. 2022Voice Filter: Few-shot text-to-speech speaker adaptation using voice conversion as a post-processing modulecited by 8
  7. 2020Latent linguistic embedding for cross-lingual text-to-speech and voice conversioncited by 4
  8. 2022GlowVC: Mel-spectrogram space disentangling model for language-independent text-free voice conversioncited by 2

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 1.

3 papers.

  1. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
  2. 2023Cross-lingual Knowledge Distillation via Flow-based Voice Conversion for Robust Polyglot Text-To-Speechcited by 0
  3. 2025Unseen Speaker and Language Adaptation for Lightweight Text-To-Speech with Adapterscited by 0