# DART: Disentanglement of Accent and Speaker Representation in Multispeaker Text-to-Speech

arXiv: https://arxiv.org/abs/2410.13342 · HTML: https://mlsanity.com/paper/2410.13342/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 20 of 20)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning](https://mlsanity.com/paper/2008.03648/) (2020)
- [Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations](https://mlsanity.com/paper/1705.08841/) (2017)
- [Hierarchical Generative Modeling for Controllable Speech Synthesis](https://mlsanity.com/paper/1810.07217/) (2018)
- [Deep Voice 2: Multi-Speaker Neural Text-to-Speech](https://mlsanity.com/paper/1705.08947/) (2017)
- [A Review of Deep Learning Techniques for Speech Processing](https://mlsanity.com/paper/2305.00359/) (2023)
- [MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesis](https://mlsanity.com/paper/2201.06460/) (2022)
- [Accent and Speaker Disentanglement in Many-to-many Voice Conversion](https://mlsanity.com/paper/2011.08609/) (2020)
- [TTS-Guided Training for Accent Conversion Without Parallel Data](https://mlsanity.com/paper/2212.10204/) (2022)
- [ECAPA-TDNN for Multi-speaker Text-to-speech Synthesis](https://mlsanity.com/paper/2203.10473/) (2022)
- [Accented Text-to-Speech Synthesis with a Conditional Variational Autoencoder](https://mlsanity.com/paper/2211.03316/) (2022)

## Built on by (most cited first, 3 of 3)

- [WenetSpeech-Chuan: A Large-Scale Sichuanese Corpus with Rich Annotation for Dialectal Speech Processing](https://mlsanity.com/paper/2509.18004/) (2025)
- [Pairwise Evaluation of Accent Similarity in Speech Synthesis](https://mlsanity.com/paper/2505.14410/) (2025)
- [Extracting accent features in spoken Brazilian Portuguese without sociolinguistic labels](https://mlsanity.com/paper/2605.30457/) (2026)

## Abstract (arXiv)

Recent advancements in Text-to-Speech (TTS) systems have enabled the generation of natural and expressive speech from textual input. Accented TTS aims to enhance user experience by making the synthesized speech more relatable to minority group listeners, and useful across various applications and context. Speech synthesis can further be made more flexible by allowing users to choose any combination of speaker identity and accent, resulting in a wide range of personalized speech outputs. Current models struggle to disentangle speaker and accent representation, making it difficult to accurately imitate different accents while maintaining the same speaker characteristics. We propose a novel approach to disentangle speaker and accent representations using multi-level variational autoencoders (ML-VAE) and vector quantization (VQ) to improve flexibility and enhance personalization in speech synthesis. Our proposed method addresses the challenge of effectively separating speaker and accent characteristics, enabling more fine-grained control over the synthesized speech. Code and speech samples are publicly available.