# XEmoRAG: Cross-Lingual Emotion Transfer with Controllable Intensity Using Retrieval-Augmented Generation arXiv: https://arxiv.org/abs/2508.07302 ยท HTML: https://mlsanity.com/paper/2508.07302/ ## Key facts - Cited by 1 papers in the map (data through 20 Oct 2026). - Placed in Emotional speech synthesis and recognition, one of 11 directions in Speech and audio language models. - Builds on 26 of its 29 references that are in the map. - First posted on arXiv 10 Aug 2025. ## Placed in - Speech and audio language models / [Emotional speech synthesis and recognition](https://mlsanity.com/speech/emotional-speech-synthesis-and-recognition/) ## Builds on (most cited first, 26 of 26) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020) - [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021) - [GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot](https://mlsanity.com/paper/2412.02612/) (2024) - [XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model](https://mlsanity.com/paper/2406.04904/) (2024) - [MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer](https://mlsanity.com/paper/2409.00750/) (2024) - [CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training](https://mlsanity.com/paper/2505.17589/) (2025) - [Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling](https://mlsanity.com/paper/2303.03926/) (2023) - [Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit](https://mlsanity.com/paper/2210.17016/) (2022) - [AdaSpeech: Adaptive Text to Speech for Custom Voice](https://mlsanity.com/paper/2103.00993/) (2021) - [Zipformer: A faster and better encoder for automatic speech recognition](https://mlsanity.com/paper/2310.11230/) (2023) - [Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model](https://mlsanity.com/paper/2408.17175/) (2024) - [Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis](https://mlsanity.com/paper/2502.04128/) (2025) - [Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloning](https://mlsanity.com/paper/1907.04448/) (2019) - [DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021](https://mlsanity.com/paper/2110.12612/) (2021) - [WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models](https://mlsanity.com/paper/2502.14727/) (2025) - [iEmoTTS: Toward Robust Cross-Speaker Emotion Transfer and Control for Speech Synthesis based on Disentanglement between Prosody and Timbre](https://mlsanity.com/paper/2206.14866/) (2022) - [ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Models](https://mlsanity.com/paper/2305.13831/) (2023) - [METTS: Multilingual Emotional Text-to-Speech by Cross-speaker and Cross-lingual Emotion Transfer](https://mlsanity.com/paper/2307.15951/) (2023) - [Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models](https://mlsanity.com/paper/2412.13702/) (2024) - [DSPGAN: a GAN-based universal vocoder for high-fidelity TTS by time-frequency domain supervision from DSP](https://mlsanity.com/paper/2211.01087/) (2022) - [DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech](https://mlsanity.com/paper/2306.14145/) (2023) - [DiCLET-TTS: Diffusion Model based Cross-lingual Emotion Transfer for Text-to-Speech -- A Study between English and Mandarin](https://mlsanity.com/paper/2309.00883/) (2023) - [AutoStyle-TTS: Retrieval-Augmented Generation based Automatic Style Matching Text-to-Speech Synthesis](https://mlsanity.com/paper/2504.10309/) (2025) - [Zero-Shot Emotion Transfer For Cross-Lingual Speech Synthesis](https://mlsanity.com/paper/2310.03963/) (2023) - [Zero-shot Cross-lingual Voice Transfer for TTS](https://mlsanity.com/paper/2409.13910/) (2024) ## Built on by (most cited first, 1 of 1) - [Towards Building Speech Large Language Models for Multitask Understanding in Low-Resource Languages](https://mlsanity.com/paper/2509.14804/) (2025) ## Abstract (arXiv) Zero-shot emotion transfer in cross-lingual speech synthesis refers to generating speech in a target language, where the emotion is expressed based on reference speech from a different source language. However, this task remains challenging due to the scarcity of parallel multilingual emotional corpora, the presence of foreign accent artifacts, and the difficulty of separating emotion from language-specific prosodic features. In this paper, we propose XEmoRAG, a novel framework to enable zero-shot emotion transfer from Chinese to Thai using a large language model (LLM)-based model, without relying on parallel emotional data. XEmoRAG extracts language-agnostic emotional embeddings from Chinese speech and retrieves emotionally matched Thai utterances from a curated emotional database, enabling controllable emotion transfer without explicit emotion labels. Additionally, a flow-matching alignment module minimizes pitch and duration mismatches, ensuring natural prosody. It also blends Chinese timbre into the Thai synthesis, enhancing rhythmic accuracy and emotional expression, while preserving speaker characteristics and emotional consistency. Experimental results show that XEmoRAG synthesizes expressive and natural Thai speech using only Chinese reference audio, without requiring explicit emotion labels. These results highlight XEmoRAG's capability to achieve flexible and low-resource emotional transfer across languages. Our demo is available at https://tlzuo-lesley.github.io/Demo-page/ .