# Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis

arXiv: https://arxiv.org/abs/2104.06835 · HTML: https://mlsanity.com/paper/2104.06835/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 14 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks](https://mlsanity.com/paper/1909.01315/) (2019)
- [Do Attention Heads in BERT Track Syntactic Dependencies?](https://mlsanity.com/paper/1911.12246/) (2019)
- [Learning latent representations for style control and transfer in end-to-end speech synthesis](https://mlsanity.com/paper/1812.04342/) (2018)
- [Exploiting Syntactic Features in a Parsed Tree to Improve End-to-End TTS](https://mlsanity.com/paper/1904.04764/) (2019)
- [GraphSpeech: Syntax-Aware Graph Attention Network For Neural Speech Synthesis](https://mlsanity.com/paper/2010.12423/) (2020)
- [A Survey of Syntactic-Semantic Parsing Based on Constituent and Dependency Structures](https://mlsanity.com/paper/2006.11056/) (2020)
- [GraphTTS: graph-to-sequence modelling in neural text-to-speech](https://mlsanity.com/paper/2003.01924/) (2020)
- [Syntactic representation learning for neural network based TTS with syntactic parse tree traversal](https://mlsanity.com/paper/2012.06971/) (2020)

## Built on by (most cited first, 3 of 3)

- [Controlling Emotion in Text-to-Speech with Natural Language Prompts](https://mlsanity.com/paper/2406.06406/) (2024)
- [RWEN-TTS: Relation-aware Word Encoding Network for Natural Text-to-Speech Synthesis](https://mlsanity.com/paper/2212.07939/) (2022)
- [Quantification and Validation for Degree of Understanding in M2M Semantic Communications](https://mlsanity.com/paper/2408.00767/) (2024)

## Abstract (arXiv)

Exploiting rich linguistic information in raw text is crucial for expressive text-to-speech (TTS). As large scale pre-trained text representation develops, bidirectional encoder representations from Transformers (BERT) has been proven to embody semantic information and employed to TTS recently. However, original or simply fine-tuned BERT embeddings still cannot provide sufficient semantic knowledge that expressive TTS models should take into account. In this paper, we propose a word-level semantic representation enhancing method based on dependency structure and pre-trained BERT embedding. The BERT embedding of each word is reprocessed considering its specific dependencies and related words in the sentence, to generate more effective semantic representation for TTS. To better utilize the dependency structure, relational gated graph network (RGGN) is introduced to make semantic information flow and aggregate through the dependency structure. The experimental results show that the proposed method can further improve the naturalness and expressiveness of synthesized speeches on both Mandarin and English datasets.