# Hierarchical Prosody Modeling for Non-Autoregressive Speech Synthesis

arXiv: https://arxiv.org/abs/2011.06465 · HTML: https://mlsanity.com/paper/2011.06465/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2020.

## Built on by (most cited first, 16 of 16)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [ProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speech](https://mlsanity.com/paper/2202.07816/) (2022)
- [Hierarchical prosody modeling and control in non-autoregressive parallel neural TTS](https://mlsanity.com/paper/2110.02952/) (2021)
- [CorrTalk: Correlation Between Hierarchical Speech and Facial Activity Variances for 3D Animation](https://mlsanity.com/paper/2310.11295/) (2023)
- [Unsupervised word-level prosody tagging for controllable speech synthesis](https://mlsanity.com/paper/2202.07200/) (2022)
- [ADAPTERMIX: Exploring the Efficacy of Mixture of Adapters for Low-Resource TTS Adaptation](https://mlsanity.com/paper/2305.18028/) (2023)
- [Voice Cloning: Comprehensive Survey](https://mlsanity.com/paper/2505.00579/) (2025)
- [Improved Prosodic Clustering for Multispeaker and Speaker-independent Phoneme-level Prosody Control](https://mlsanity.com/paper/2111.10168/) (2021)
- [Controllable speech synthesis by learning discrete phoneme-level prosodic representations](https://mlsanity.com/paper/2211.16307/) (2022)
- [Improving Prosody for Unseen Texts in Speech Synthesis by Utilizing Linguistic Information and Noisy Data](https://mlsanity.com/paper/2111.07549/) (2021)
- [UniTTS: Residual Learning of Unified Embedding Space for Speech Style Control](https://mlsanity.com/paper/2106.11171/) (2021)
- [ProMode: A Speech Prosody Model Conditioned on Acoustic and Textual Inputs](https://mlsanity.com/paper/2508.09389/) (2025)
- [Multi-Scale Spectrogram Modelling for Neural Text-to-Speech](https://mlsanity.com/paper/2106.15649/) (2021)
- [Predicting phoneme-level prosody latents using AR and flow-based Prior Networks for expressive speech synthesis](https://mlsanity.com/paper/2211.01327/) (2022)
- [KEST: Kernel Distance Based Efficient Self-Training for Improving Controllable Text Generation](https://mlsanity.com/paper/2306.10414/) (2023)

## Abstract (arXiv)

Prosody modeling is an essential component in modern text-to-speech (TTS) frameworks. By explicitly providing prosody features to the TTS model, the style of synthesized utterances can thus be controlled. However, predicting natural and reasonable prosody at inference time is challenging. In this work, we analyzed the behavior of non-autoregressive TTS models under different prosody-modeling settings and proposed a hierarchical architecture, in which the prediction of phoneme-level prosody features are conditioned on the word-level prosody features. The proposed method outperforms other competitors in terms of audio quality and prosody naturalness in our objective and subjective evaluation.