# Latent linguistic embedding for cross-lingual text-to-speech and voice conversion

arXiv: https://arxiv.org/abs/2010.03717 · HTML: https://mlsanity.com/paper/2010.03717/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 4 of 4)

- [Cross-lingual Low Resource Speaker Adaptation Using Phonological Features](https://mlsanity.com/paper/2111.09075/) (2021)
- [Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciation](https://mlsanity.com/paper/2210.17264/) (2022)
- [Low-Latency Real-Time Non-Parallel Voice Conversion based on Cyclic Variational Autoencoder and Multiband WaveRNN with Data-Driven Linear Prediction](https://mlsanity.com/paper/2105.09858/) (2021)
- [Preliminary study on using vector quantization latent spaces for TTS/VC systems with consistent performance](https://mlsanity.com/paper/2106.13479/) (2021)

## Abstract (arXiv)

As the recently proposed voice cloning system, NAUTILUS, is capable of cloning unseen voices using untranscribed speech, we investigate the feasibility of using it to develop a unified cross-lingual TTS/VC system. Cross-lingual speech generation is the scenario in which speech utterances are generated with the voices of target speakers in a language not spoken by them originally. This type of system is not simply cloning the voice of the target speaker, but essentially creating a new voice that can be considered better than the original under a specific framing. By using a well-trained English latent linguistic embedding to create a cross-lingual TTS and VC system for several German, Finnish, and Mandarin speakers included in the Voice Conversion Challenge 2020, we show that our method not only creates cross-lingual VC with high speaker similarity but also can be seamlessly used for cross-lingual TTS without having to perform any extra steps. However, the subjective evaluations of perceived naturalness seemed to vary between target speakers, which is one aspect for future improvement.