# AccentBox: Towards High-Fidelity Zero-Shot Accent Generation

arXiv: https://arxiv.org/abs/2409.09098 · HTML: https://mlsanity.com/paper/2409.09098/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 19 of 19)

- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models](https://mlsanity.com/paper/2403.03100/) (2024)
- [Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision](https://mlsanity.com/paper/2302.03540/) (2023)
- [VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2406.05370/) (2024)
- [LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus](https://mlsanity.com/paper/2305.18802/) (2023)
- [Natural language guidance of high-fidelity text-to-speech with synthetic annotations](https://mlsanity.com/paper/2402.01912/) (2024)
- [The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASR](https://mlsanity.com/paper/2303.18110/) (2023)
- [The Accented English Speech Recognition Challenge 2020: Open Datasets, Tracks, Baselines, Results and Methods](https://mlsanity.com/paper/2102.10233/) (2021)
- [CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voice](https://mlsanity.com/paper/2305.18283/) (2023)
- [Accented Text-to-Speech Synthesis with Limited Data](https://mlsanity.com/paper/2305.04816/) (2023)
- [Explicit Intensity Control for Accented Text-to-speech](https://mlsanity.com/paper/2210.15364/) (2022)
- [Accent-VITS:accent transfer for end-to-end TTS](https://mlsanity.com/paper/2312.16850/) (2023)

## Built on by (most cited first, 9 of 9)

- [MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis](https://mlsanity.com/paper/2502.18924/) (2025)
- [Voxlect: A Speech Foundation Model Benchmark for Modeling Dialects and Regional Languages Around the Globe](https://mlsanity.com/paper/2508.01691/) (2025)
- [Pairwise Evaluation of Accent Similarity in Speech Synthesis](https://mlsanity.com/paper/2505.14410/) (2025)
- [Quantifying Speaker Embedding Phonological Rule Interactions in Accented Speech Synthesis](https://mlsanity.com/paper/2601.14417/) (2026)
- [Rethinking Discrete Speech Representation Tokens for Accent Generation](https://mlsanity.com/paper/2601.19786/) (2026)
- [An Evaluation Framework for Text-to-Speech Voice Reconstruction](https://mlsanity.com/paper/2606.21343/) (2026)
- [Synthetic Audio Generation Framework for Air Traffic Control Speech Recognition](https://mlsanity.com/paper/2606.21340/) (2026)
- [Flexible and Interpretable Accent Distance Measurements](https://mlsanity.com/paper/2609.11458/) (2026)
- [A Cross Community Agenda for Speech AI](https://mlsanity.com/paper/2609.13168/) (2026)

## Abstract (arXiv)

While recent Zero-Shot Text-to-Speech (ZS-TTS) models have achieved high naturalness and speaker similarity, they fall short in accent fidelity and control. To address this issue, we propose zero-shot accent generation that unifies Foreign Accent Conversion (FAC), accented TTS, and ZS-TTS, with a novel two-stage pipeline. In the first stage, we achieve state-of-the-art (SOTA) on Accent Identification (AID) with 0.56 f1 score on unseen speakers. In the second stage, we condition a ZS-TTS system on the pretrained speaker-agnostic accent embeddings extracted by the AID model. The proposed system achieves higher accent fidelity on inherent/cross accent generation, and enables unseen accent generation.