A Discourse-level Multi-scale Prosodic Model for Fine-grained Emotion Analysis
Not on the map. Why?
Abstract (arXiv)
This paper explores predicting suitable prosodic features for fine-grained emotion analysis from the discourse-level text. To obtain fine-grained emotional prosodic features as predictive values for our model, we extract a phoneme-level Local Prosody Embedding sequence (LPEs) and a Global Style Embedding as prosodic speech features from the speech with the help of a style transfer model. We propose a Discourse-level Multi-scale text Prosodic Model (D-MPM) that exploits multi-scale text to predict these two prosodic features. The proposed model can be used to analyze better emotional prosodic features and thus guide the speech synthesis model to synthesize more expressive speech. To quantitatively evaluate the proposed model, we contribute a new and large-scale Discourse-level Chinese Audiobook (DCA) dataset with more than 13,000 utterances annotated sequences to evaluate the proposed model. Experimental results on the DCA dataset show that the multi-scale text information effectively helps to predict prosodic features, and the discourse-level text improves both the overall coherence and the user experience. More interestingly, although we aim at the synthesis effect of the style transfer model, the synthesized speech by the proposed text prosodic analysis model is even better than the style transfer from the original speech in some user evaluation indicators.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 25 of its 34 references that are in the map. First posted on arXiv 21 Sep 2023.
Builds on
25 of 34 references.
References · 25
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2018Deep contextualized word representationscited by 2,285
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2020Revisiting Pre-Trained Models for Chinese Natural Language Processingcited by 262
- 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
- 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
- 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
- 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
- 2017Deep Voice: Real-time Neural Text-to-Speechcited by 83
13 more
- 2018Robust and fine-grained prosody control of end-to-end speech synthesiscited by 48
- 2019Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokenscited by 44
- 2020Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesiscited by 41
- 2018Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesiscited by 31
- 2020Fine-grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesiscited by 29
- 2017Multiple Instance Learning Networks for Fine-Grained Sentiment Analysiscited by 23
- 2021Towards Multi-Scale Style Control for Expressive Speech Synthesiscited by 18
- 2019Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehensioncited by 12
- 2019Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistencycited by 12
- 2019Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness Featurescited by 10
- 2020Controllable Neural Prosody Synthesiscited by 10
- 2020Hierarchical Multi-Grained Generative Model for Expressive Speech Synthesiscited by 10
- 2021Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarioscited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.