# Accented Text-to-Speech Synthesis with Limited Data

arXiv: https://arxiv.org/abs/2305.04816 · HTML: https://mlsanity.com/paper/2305.04816/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2023.

## Built on by (most cited first, 12 of 12)

- [CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech](https://mlsanity.com/paper/2506.02863/) (2025)
- [AccentBox: Towards High-Fidelity Zero-Shot Accent Generation](https://mlsanity.com/paper/2409.09098/) (2024)
- [Forensic deepfake audio detection using segmental speech features](https://mlsanity.com/paper/2505.13847/) (2025)
- [Pairwise Evaluation of Accent Similarity in Speech Synthesis](https://mlsanity.com/paper/2505.14410/) (2025)
- [Quantifying Speaker Embedding Phonological Rule Interactions in Accented Speech Synthesis](https://mlsanity.com/paper/2601.14417/) (2026)
- [Non-autoregressive real-time Accent Conversion model with voice cloning](https://mlsanity.com/paper/2405.13162/) (2024)
- [Learning-free L2-Accented Speech Generation using Phonological Rules](https://mlsanity.com/paper/2603.07550/) (2026)
- [CrossAccent-TTS: Cross-Lingual Accent-Intensity Controllable Text-to-Speech via Disentangled Speaker and Accent Representations](https://mlsanity.com/paper/2606.25403/) (2026)
- [Optimizing Multilingual Text-To-Speech with Accents & Emotions](https://mlsanity.com/paper/2506.16310/) (2025)
- [Mic Drop or Data Flop? Evaluating the Fitness for Purpose of AI Voice Interviewers for Data Collection within Quantitative & Qualitative Research Contexts](https://mlsanity.com/paper/2509.01814/) (2025)
- [ASTRA: A Scalable Next-Generation ATCO Training Simulator with Autonomous Simpilots](https://mlsanity.com/paper/2606.18319/) (2026)
- [A survey of AI-generated voices and their detection](https://mlsanity.com/paper/2608.15411/) (2026)

## Abstract (arXiv)

This paper presents an accented text-to-speech (TTS) synthesis framework with limited training data. We study two aspects concerning accent rendering: phonetic (phoneme difference) and prosodic (pitch pattern and phoneme duration) variations. The proposed accented TTS framework consists of two models: an accented front-end for grapheme-to-phoneme (G2P) conversion and an accented acoustic model with integrated pitch and duration predictors for phoneme-to-Mel-spectrogram prediction. The accented front-end directly models the phonetic variation, while the accented acoustic model explicitly controls the prosodic variation. Specifically, both models are first pre-trained on a large amount of data, then only the accent-related layers are fine-tuned on a limited amount of data for the target accent. In the experiments, speech data of three English accents, i.e., General American English, Irish English, and British English Received Pronunciation, are used for pre-training. The pre-trained models are then fine-tuned with Scottish and General Australian English accents, respectively. Both objective and subjective evaluation results show that the accented TTS front-end fine-tuned with a small accented phonetic lexicon (5k words) effectively handles the phonetic variation of accents, while the accented TTS acoustic model fine-tuned with a limited amount of accented speech data (approximately 3 minutes) effectively improves the prosodic rendering including pitch and duration. The overall accent modeling contributes to improved speech quality and accent similarity.