# JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment

arXiv: https://arxiv.org/abs/2005.07799 · HTML: https://mlsanity.com/paper/2005.07799/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2020.

## Built on by (most cited first, 13 of 13)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search](https://mlsanity.com/paper/2102.04040/) (2021)
- [JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speech](https://mlsanity.com/paper/2203.16852/) (2022)
- [DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Dataset](https://mlsanity.com/paper/2409.08731/) (2024)
- [Virtuoso: Massive Multilingual Speech-Text Joint Semi-Supervised Learning for Text-To-Speech](https://mlsanity.com/paper/2210.15447/) (2022)
- [VNet: A GAN-based Multi-Tier Discriminator Network for Speech Synthesis Vocoders](https://mlsanity.com/paper/2408.06906/) (2024)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [A Survey of Deep Learning Audio Generation Methods](https://mlsanity.com/paper/2406.00146/) (2024)
- [FastFit: Towards Real-Time Iterative Neural Vocoder by Replacing U-Net Encoder With Multiple STFTs](https://mlsanity.com/paper/2305.10823/) (2023)
- [Triple M: A Practical Text-to-speech Synthesis System With Multi-guidance Attention And Multi-band Multi-time LPCNet](https://mlsanity.com/paper/2102.00247/) (2021)
- [PAMA-TTS: Progression-Aware Monotonic Attention for Stable Seq2Seq TTS With Accurate Phoneme Duration Control](https://mlsanity.com/paper/2110.04486/) (2021)
- [AutoTTS: End-to-End Text-to-Speech Synthesis through Differentiable Duration Modeling](https://mlsanity.com/paper/2203.11049/) (2022)

## Abstract (arXiv)

We propose Jointly trained Duration Informed Transformer (JDI-T), a feed-forward Transformer with a duration predictor jointly trained without explicit alignments in order to generate an acoustic feature sequence from an input text. In this work, inspired by the recent success of the duration informed networks such as FastSpeech and DurIAN, we further simplify its sequential, two-stage training pipeline to a single-stage training. Specifically, we extract the phoneme duration from the autoregressive Transformer on the fly during the joint training instead of pretraining the autoregressive model and using it as a phoneme duration extractor. To our best knowledge, it is the first implementation to jointly train the feed-forward Transformer without relying on a pre-trained phoneme duration extractor in a single training pipeline. We evaluate the effectiveness of the proposed model on the publicly available Korean Single speaker Speech (KSS) dataset compared to the baseline text-to-speech (TTS) models trained by ESPnet-TTS.