# AutoTTS: End-to-End Text-to-Speech Synthesis through Differentiable Duration Modeling

arXiv: https://arxiv.org/abs/2203.11049 · HTML: https://mlsanity.com/paper/2203.11049/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search](https://mlsanity.com/paper/2005.11129/) (2020)
- [Deep Voice: Real-time Neural Text-to-Speech](https://mlsanity.com/paper/1702.07825/) (2017)
- [End-to-End Adversarial Text-to-Speech](https://mlsanity.com/paper/2006.03575/) (2020)
- [Deep Voice 2: Multi-Speaker Neural Text-to-Speech](https://mlsanity.com/paper/1705.08947/) (2017)
- [Parallel Tacotron: Non-Autoregressive and Controllable TTS](https://mlsanity.com/paper/2010.11439/) (2020)
- [Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS](https://mlsanity.com/paper/1906.00672/) (2019)
- [Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling](https://mlsanity.com/paper/2103.14574/) (2021)
- [AlignTTS: Efficient Feed-Forward Text-to-Speech System without Explicit Alignment](https://mlsanity.com/paper/2003.01950/) (2020)
- [SpeedySpeech: Efficient Neural Speech Synthesis](https://mlsanity.com/paper/2008.03802/) (2020)
- [EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecture](https://mlsanity.com/paper/2012.03500/) (2020)
- [JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment](https://mlsanity.com/paper/2005.07799/) (2020)

## Abstract (arXiv)

Parallel text-to-speech (TTS) models have recently enabled fast and highly-natural speech synthesis. However, they typically require external alignment models, which are not necessarily optimized for the decoder as they are not jointly trained. In this paper, we propose a differentiable duration method for learning monotonic alignments between input and output sequences. Our method is based on a soft-duration mechanism that optimizes a stochastic process in expectation. Using this differentiable duration method, we introduce AutoTTS, a direct text-to-waveform speech synthesis model. AutoTTS enables high-fidelity speech synthesis through a combination of adversarial training and matching the total ground-truth duration. Experimental results show that our model obtains competitive results while enjoying a much simpler training pipeline. Audio samples are available online.