CrossSpeech: Speaker-independent Acoustic Representation for Cross-lingual Speech Synthesis
Not on the map. Why?
Abstract (arXiv)
While recent text-to-speech (TTS) systems have made remarkable strides toward human-level quality, the performance of cross-lingual TTS lags behind that of intra-lingual TTS. This gap is mainly rooted from the speaker-language entanglement problem in cross-lingual TTS. In this paper, we propose CrossSpeech which improves the quality of cross-lingual speech by effectively disentangling speaker and language information in the level of acoustic feature space. Specifically, CrossSpeech decomposes the speech generation pipeline into the speaker-independent generator (SIG) and speaker-dependent generator (SDG). The SIG produces the speaker-independent acoustic representation which is not biased to specific speaker distributions. On the other hand, the SDG models speaker-dependent speech variation that characterizes speaker attributes. By handling each information separately, CrossSpeech can obtain disentangled speaker and language representations. From the experiments, we verify that CrossSpeech achieves significant improvements in cross-lingual TTS, especially in terms of speaker similarity to the target speaker.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 17 of its 28 references that are in the map. First posted on arXiv 28 Feb 2023.
Builds on
17 of 28 references.
References · 17
- 2017Attention Is All You Needcited by 60,361
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 674
- 2021Domain Generalization with MixStylecited by 566
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
- 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
- 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
- 2019Feature-Critic Networks for Heterogeneous Domain Generalizationcited by 130
- 2019Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloningcited by 66
- 2018Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytescited by 54
- 2022Pushing the limits of raw waveform speaker recognitioncited by 51
- 2021One TTS Alignment To Rule Them Allcited by 47
5 more
- 2021Fre-GAN: Adversarial Frequency-consistent Audio Synthesiscited by 24
- 2022GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speechcited by 21
- 2020Phonological Features for 0-shot Multilingual Speech Synthesiscited by 20
- 2022Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Featurescited by 20
- 2022Heterogeneous Target Speech Separationcited by 13
Built on by
Citing papers per year: 2024: 2; 2025: 1.
3 papers.