# End-to-End Text-to-Speech using Latent Duration based on VQ-VAE

arXiv: https://arxiv.org/abs/2010.09602 · HTML: https://mlsanity.com/paper/2010.09602/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 7 of 7)

- [A Complexity-Based Theory of Compositionality](https://mlsanity.com/paper/2410.14817/) (2024)
- [U-DiT TTS: U-Diffusion Vision Transformer for Text-to-Speech](https://mlsanity.com/paper/2305.13195/) (2023)
- [Neural HMMs are all you need (for high-quality attention-free TTS)](https://mlsanity.com/paper/2108.13320/) (2021)
- [Variational latent discrete representation for time series modelling](https://mlsanity.com/paper/2306.15282/) (2023)
- [Cross-Utterance Conditioned VAE for Non-Autoregressive Text-to-Speech](https://mlsanity.com/paper/2205.04120/) (2022)
- [Cross-Utterance Conditioned VAE for Speech Generation](https://mlsanity.com/paper/2309.04156/) (2023)
- [Comparative Evaluation of Expressive Japanese Character Text-to-Speech with VITS and Style-BERT-VITS2](https://mlsanity.com/paper/2505.17320/) (2025)

## Abstract (arXiv)

Explicit duration modeling is a key to achieving robust and efficient alignment in text-to-speech synthesis (TTS). We propose a new TTS framework using explicit duration modeling that incorporates duration as a discrete latent variable to TTS and enables joint optimization of whole modules from scratch. We formulate our method based on conditional VQ-VAE to handle discrete duration in a variational autoencoder and provide a theoretical explanation to justify our method. In our framework, a connectionist temporal classification (CTC) -based force aligner acts as the approximate posterior, and text-to-duration works as the prior in the variational autoencoder. We evaluated our proposed method with a listening test and compared it with other TTS methods based on soft-attention or explicit duration modeling. The results showed that our systems rated between soft-attention-based methods (Transformer-TTS, Tacotron2) and explicit duration modeling-based methods (Fastspeech).