UtterTune: LoRA-Based Target-Language Pronunciation Edit and Control in Multilingual Text-to-Speech
Not on the map. Why?
Abstract (arXiv)
We propose UtterTune, a lightweight method for adapting a multilingual text-to-speech (TTS) system built on a large language model (LLM). It improves control of pronunciation in the target language while preserving performance in the others. Although LLM architectures have enabled TTS models to achieve remarkable naturalness, accurately modeling grapheme-to-phoneme (G2P) mapping and prosody remains challenging, especially when the model omits an explicit G2P module and directly processes minimally encoded text (e.g., byte-pair encoding). UtterTune leverages low-rank adaptation to enable the control of segmental pronunciation and pitch accent at the phoneme level for Japanese speech, the target language in this paper, while maintaining naturalness and speaker similarity in a zero-shot setting. Objective and subjective evaluations confirm its effectiveness.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 42 references that are in the map. First posted on arXiv 13 Aug 2025.
Builds on
16 of 42 references.
References · 16
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
- 2024Qwen2.5-Coder Technical Reportcited by 1,203
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
- 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 169
- 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 122
- 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
- 2017JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesiscited by 58
- 2023An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verificationcited by 42
- 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 22
4 more
- 2022Adapter-Based Extension of Multi-Speaker Text-to-Speech Model for New Speakerscited by 9
- 2024StyleSpeech: Parameter-efficient Fine Tuning for Pre-trained Controllable Text-to-Speechcited by 4
- 2025LoRP-TTS: Low-Rank Personalized Text-To-Speechcited by 3
- 2024VoiceTailor: Lightweight Plug-In Adapter for Diffusion-Based Personalized Text-to-Speechcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.