# Cross-speaker Emotion Transfer Based on Speaker Condition Layer Normalization and Semi-Supervised Training in Text-To-Speech

arXiv: https://arxiv.org/abs/2110.04153 · HTML: https://mlsanity.com/paper/2110.04153/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2021.

## Built on by (most cited first, 8 of 8)

- [ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Models](https://mlsanity.com/paper/2305.13831/) (2023)
- [Style-Label-Free: Cross-Speaker Style Transfer by Quantized VAE and Speaker-wise Normalization in Speech Synthesis](https://mlsanity.com/paper/2212.06397/) (2022)
- [Improving Prosody for Cross-Speaker Style Transfer by Semi-Supervised Style Extractor and Hierarchical Modeling in Speech Synthesis](https://mlsanity.com/paper/2303.07711/) (2023)
- [M2-CTTS: End-to-End Multi-scale Multi-modal Conversational Text-to-Speech Synthesis](https://mlsanity.com/paper/2305.02269/) (2023)
- [LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance](https://mlsanity.com/paper/2406.05325/) (2024)
- [UMETTS: A Unified Framework for Emotional Text-to-Speech Synthesis with Multimodal Prompts](https://mlsanity.com/paper/2404.18398/) (2024)
- [LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency Distillation](https://mlsanity.com/paper/2408.12354/) (2024)
- [Towards Cross-speaker Reading Style Transfer on Audiobook Dataset](https://mlsanity.com/paper/2208.05359/) (2022)

## Abstract (arXiv)

In expressive speech synthesis, there are high requirements for emotion interpretation. However, it is time-consuming to acquire emotional audio corpus for arbitrary speakers due to their deduction ability. In response to this problem, this paper proposes a cross-speaker emotion transfer method that can realize the transfer of emotions from source speaker to target speaker. A set of emotion tokens is firstly defined to represent various categories of emotions. They are trained to be highly correlated with corresponding emotions for controllable synthesis by cross-entropy loss and semi-supervised training strategy. Meanwhile, to eliminate the down-gradation to the timbre similarity from cross-speaker emotion transfer, speaker condition layer normalization is implemented to model speaker characteristics. Experimental results show that the proposed method outperforms the multi-reference based baseline in terms of timbre similarity, stability and emotion perceive evaluations.