# Ensemble prosody prediction for expressive speech synthesis

arXiv: https://arxiv.org/abs/2304.00714 · HTML: https://mlsanity.com/paper/2304.00714/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2023.

## Built on by (most cited first, 2 of 2)

- [Style Mixture of Experts for Expressive Text-To-Speech Synthesis](https://mlsanity.com/paper/2406.03637/) (2024)
- [Gesture2Speech: How Far Can Hand Movements Shape Expressive Speech?](https://mlsanity.com/paper/2603.19831/) (2026)

## Abstract (arXiv)

Generating expressive speech with rich and varied prosody continues to be a challenge for Text-to-Speech. Most efforts have focused on sophisticated neural architectures intended to better model the data distribution. Yet, in evaluations it is generally found that no single model is preferred for all input texts. This suggests an approach that has rarely been used before for Text-to-Speech: an ensemble of models. We apply ensemble learning to prosody prediction. We construct simple ensembles of prosody predictors by varying either model architecture or model parameter values. To automatically select amongst the models in the ensemble when performing Text-to-Speech, we propose a novel, and computationally trivial, variance-based criterion. We demonstrate that even a small ensemble of prosody predictors yields useful diversity, which, combined with the proposed selection criterion, outperforms any individual model from the ensemble.