# Towards Multi-Scale Speaking Style Modelling with Hierarchical Context Information for Mandarin Speech Synthesis

arXiv: https://arxiv.org/abs/2204.02743 · HTML: https://mlsanity.com/paper/2204.02743/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2022.

## Built on by (most cited first, 4 of 4)

- [CONCSS: Contrastive-based Context Comprehension for Dialogue-appropriate Prosody in Conversational Speech Synthesis](https://mlsanity.com/paper/2312.10358/) (2023)
- [StyleSpeech: Self-supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech Synthesis](https://mlsanity.com/paper/2312.12181/) (2023)
- [Towards Expressive Video Dubbing with Multiscale Multimodal Context Interaction](https://mlsanity.com/paper/2412.18748/) (2024)
- [ProMode: A Speech Prosody Model Conditioned on Acoustic and Textual Inputs](https://mlsanity.com/paper/2508.09389/) (2025)

## Abstract (arXiv)

Previous works on expressive speech synthesis focus on modelling the mono-scale style embedding from the current sentence or context, but the multi-scale nature of speaking style in human speech is neglected. In this paper, we propose a multi-scale speaking style modelling method to capture and predict multi-scale speaking style for improving the naturalness and expressiveness of synthetic speech. A multi-scale extractor is proposed to extract speaking style embeddings at three different levels from the ground-truth speech, and explicitly guide the training of a multi-scale style predictor based on hierarchical context information. Both objective and subjective evaluations on a Mandarin audiobooks dataset demonstrate that our proposed method can significantly improve the naturalness and expressiveness of the synthesized speech.