A Study of Modeling Rising Intonation in Cantonese Neural Speech Synthesis
Not on the map. Why?
Abstract (arXiv)
In human speech, the attitude of a speaker cannot be fully expressed only by the textual content. It has to come along with the intonation. Declarative questions are commonly used in daily Cantonese conversations, and they are usually uttered with rising intonation. Vanilla neural text-to-speech (TTS) systems are not capable of synthesizing rising intonation for these sentences due to the loss of semantic information. Though it has become more common to complement the systems with extra language models, their performance in modeling rising intonation is not well studied. In this paper, we propose to complement the Cantonese TTS model with a BERT-based statement/question classifier. We design different training strategies and compare their performance. We conduct our experiments on a Cantonese corpus named CanTTS. Empirical results show that the separate training approach obtains the best generalization performance and feasibility.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 3 Aug 2022.
Builds on
22 of 32 references.
References · 22
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
- 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
- 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
- 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
- 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 63
- 2018Learning latent representations for style control and transfer in end-to-end speech synthesiscited by 61
- 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
10 more
- 2017Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attentioncited by 57
- 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
- 2020Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesiscited by 27
- 2020CAMP: a Two-Stage Approach to Modelling Prosody in Contextcited by 12
- 2019Dynamic Prosody Generation for Speech Synthesis using Linguistics-Driven Acoustic Embedding Selectioncited by 11
- 2021Speech BERT Embedding For Improving Prosody in Neural TTScited by 11
- 2019Exploiting Syntactic Features in a Parsed Tree to Improve End-to-End TTScited by 10
- 2020Prosodic Representation Learning and Contextual Sampling for Neural Text-to-Speechcited by 10
- 2021Prosodic Clustering for Phoneme-level Prosody Control in End-to-End Speech Synthesiscited by 8
- 2020Syntactic representation learning for neural network based TTS with syntactic parse tree traversalcited by 4
Built on by
Citing papers per year: 2023: 2.
2 papers.