# Improving Mandarin Prosodic Structure Prediction with Multi-level Contextual Information

arXiv: https://arxiv.org/abs/2308.16577 · HTML: https://mlsanity.com/paper/2308.16577/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 31 Aug 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [DurIAN: Duration Informed Attention Network For Multimodal Synthesis](https://mlsanity.com/paper/1909.01700/) (2019)
- [Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesis](https://mlsanity.com/paper/2011.05161/) (2020)
- [Unified Mandarin TTS Front-end Based on Distilled BERT Model](https://mlsanity.com/paper/2012.15404/) (2020)
- [Transformer Language Models with LSTM-based Cross-utterance Information Representation](https://mlsanity.com/paper/2102.06474/) (2021)
- [Automatic Prosody Prediction for Chinese Speech Synthesis using BLSTM-RNN and Embedding Features](https://mlsanity.com/paper/1511.00360/) (2015)

## Built on by (most cited first, 1 of 1)

- [Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model](https://mlsanity.com/paper/2309.11000/) (2023)

## Abstract (arXiv)

For text-to-speech (TTS) synthesis, prosodic structure prediction (PSP) plays an important role in producing natural and intelligible speech. Although inter-utterance linguistic information can influence the speech interpretation of the target utterance, previous works on PSP mainly focus on utilizing intrautterance linguistic information of the current utterance only. This work proposes to use inter-utterance linguistic information to improve the performance of PSP. Multi-level contextual information, which includes both inter-utterance and intrautterance linguistic information, is extracted by a hierarchical encoder from character level, utterance level and discourse level of the input text. Then a multi-task learning (MTL) decoder predicts prosodic boundaries from multi-level contextual information. Objective evaluation results on two datasets show that our method achieves better F1 scores in predicting prosodic word (PW), prosodic phrase (PPH) and intonational phrase (IPH). It demonstrates the effectiveness of using multi-level contextual information for PSP. Subjective preference tests also indicate the naturalness of synthesized speeches are improved.