KIT's Submission to Cross-Lingual Voice Cloning in IWSLT 2026
Not on the map. Why?
Abstract (arXiv)
Cross-lingual voice cloning aims to generate speech in a target language while preserving speaker identity from a source-language reference. This task is central to speech translation and is the focus of the IWSLT 2026 Cross-Lingual Voice Cloning track. A key challenge is maintaining intelligibility and naturalness in the presence of accent variation and domain-specific vocabulary. We build on a multilingual text-to-speech model, FishAudio-S2-Pro, and introduce language tag prompting to improve language control and reduce accent leakage. We further apply reinforcement learning (RL) fine-tuning for task adaptation and observe improvements in intelligibility. Finally, we propose a reference-conditioned lexical matching method that improves pronunciation of domain-specific terms when lexical overlap is present. Results show that language prompting provides the largest gains, while lexical matching yields consistent improvements on matched subsets.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 12 references that are in the map. First posted on arXiv 5 Jun 2026.
Builds on
10 of 12 references.
References · 10
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2023Scaling Speech Technology to 1,000+ Languagescited by 411
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
- 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
- 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
- 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
- 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
- 2025Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Cross-Lingual F5-TTS: Towards Language-Agnostic Voice Cloning and Speech Synthesis
- 2026One Voice, Many Tongues: Cross-Lingual Voice Cloning for Scientific Speech
- 2026X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloning
- 2026PFluxTTS: Hybrid Flow-Matching TTS with Robust Cross-Lingual Voice Cloning and Inference-Time Model Fusion
- 2026Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder
- 2025LatinX: Aligning a Multilingual TTS Model with Direct Preference Optimization
- 2026Tacit-TTS: From Autoregressive Decoding to Masked Prediction for Efficient Transcript-Free Voice Cloning
- 2025Language translation, and change of accent for speech-to-speech task using diffusion model