# Expressive, Variable, and Controllable Duration Modelling in TTS

arXiv: https://arxiv.org/abs/2206.14165 · HTML: https://mlsanity.com/paper/2206.14165/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2022.

## Built on by (most cited first, 7 of 7)

- [Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey](https://mlsanity.com/paper/2412.06602/) (2024)
- [Duration-aware pause insertion using pre-trained language model for multi-speaker text-to-speech](https://mlsanity.com/paper/2302.13652/) (2023)
- [PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-based Prosody Modeling](https://mlsanity.com/paper/2306.07489/) (2023)
- [Controllable Emphasis with zero data for text-to-speech](https://mlsanity.com/paper/2307.07062/) (2023)
- [Comparing normalizing flows and diffusion models for prosody and acoustic modelling in text-to-speech](https://mlsanity.com/paper/2307.16679/) (2023)
- [Predicting phoneme-level prosody latents using AR and flow-based Prior Networks for expressive speech synthesis](https://mlsanity.com/paper/2211.01327/) (2022)
- [PRESENT: Zero-Shot Text-to-Prosody Control](https://mlsanity.com/paper/2408.06827/) (2024)

## Abstract (arXiv)

Duration modelling has become an important research problem once more with the rise of non-attention neural text-to-speech systems. The current approaches largely fall back to relying on previous statistical parametric speech synthesis technology for duration prediction, which poorly models the expressiveness and variability in speech. In this paper, we propose two alternate approaches to improve duration modelling. First, we propose a duration model conditioned on phrasing that improves the predicted durations and provides better modelling of pauses. We show that the duration model conditioned on phrasing improves the naturalness of speech over our baseline duration model. Second, we also propose a multi-speaker duration model called Cauliflow, that uses normalising flows to predict durations that better match the complex target duration distribution. Cauliflow performs on par with our other proposed duration model in terms of naturalness, whilst providing variable durations for the same prompt and variable levels of expressiveness. Lastly, we propose to condition Cauliflow on parameters that provide an intuitive control of the pacing and pausing in the synthesised speech in a novel way.