# Analysis and Assessment of Controllability of an Expressive Deep Learning-based TTS system

arXiv: https://arxiv.org/abs/2103.04097 · HTML: https://mlsanity.com/paper/2103.04097/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron](https://mlsanity.com/paper/1803.09047/) (2018)
- [Hierarchical Generative Modeling for Controllable Speech Synthesis](https://mlsanity.com/paper/1810.07217/) (2018)
- [Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention](https://mlsanity.com/paper/1710.08969/) (2017)
- [Fine-grained robust prosody transfer for single-speaker neural text-to-speech](https://mlsanity.com/paper/1907.02479/) (2019)
- [Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder](https://mlsanity.com/paper/1804.02135/) (2018)
- [CopyCat: Many-to-Many Fine-Grained Prosody Transfer for Neural Text-to-Speech](https://mlsanity.com/paper/2004.14617/) (2020)
- [Controllable neural text-to-speech synthesis using intuitive prosodic features](https://mlsanity.com/paper/2009.06775/) (2020)
- [Deep Encoder-Decoder Models for Unsupervised Learning of Controllable Speech Synthesis](https://mlsanity.com/paper/1807.11470/) (2018)
- [Sequence to Sequence Neural Speech Synthesis with Prosody Modification Capabilities](https://mlsanity.com/paper/1909.10302/) (2019)
- [Visualization and Interpretation of Latent Spaces for Controlling Expressive Speech Synthesis through Audio Analysis](https://mlsanity.com/paper/1903.11570/) (2019)
- [A Methodology for Controlling the Emotional Expressiveness in Synthetic Speech -- a Deep Learning approach](https://mlsanity.com/paper/1907.02784/) (2019)
- [The Theory behind Controllable Expressive Speech Synthesis: a Cross-disciplinary Approach](https://mlsanity.com/paper/1910.06234/) (2019)

## Built on by (most cited first, 4 of 4)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [MUST&P-SRL: Multi-lingual and Unified Syllabification in Text and Phonetic Domains for Speech Representation Learning](https://mlsanity.com/paper/2310.11541/) (2023)
- [TIPAA-SSL: Text Independent Phone-to-Audio Alignment based on Self-Supervised Learning and Knowledge Transfer](https://mlsanity.com/paper/2405.02124/) (2024)
- [Voice Mapping of Text-to-Speech Systems: A Metric-Based Approach for Voice Quality Assessment](https://mlsanity.com/paper/2605.00861/) (2026)

## Abstract (arXiv)

In this paper, we study the controllability of an Expressive TTS system trained on a dataset for a continuous control. The dataset is the Blizzard 2013 dataset based on audiobooks read by a female speaker containing a great variability in styles and expressiveness. Controllability is evaluated with both an objective and a subjective experiment. The objective assessment is based on a measure of correlation between acoustic features and the dimensions of the latent space representing expressiveness. The subjective assessment is based on a perceptual experiment in which users are shown an interface for Controllable Expressive TTS and asked to retrieve a synthetic utterance whose expressiveness subjectively corresponds to that a reference utterance.