# Predicting Prosodic Prominence from Text with Pre-trained Contextualized Word Representations

arXiv: https://arxiv.org/abs/1908.02262 · HTML: https://mlsanity.com/paper/1908.02262/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2019.

## Built on by (most cited first, 14 of 14)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Quantifying the redundancy between prosody and text](https://mlsanity.com/paper/2311.17233/) (2023)
- [BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model](https://mlsanity.com/paper/2207.01718/) (2022)
- [Metrical Tagging in the Wild: Building and Annotating Poetry Corpora with Rhythmic Features](https://mlsanity.com/paper/2102.08858/) (2021)
- [Deep Learning For Prominence Detection In Children's Read Speech](https://mlsanity.com/paper/2110.14273/) (2021)
- [The Role of Prosody in Spoken Question Answering](https://mlsanity.com/paper/2502.05389/) (2025)
- [Can LLMs Understand the Implication of Emphasized Sentences in Dialogue?](https://mlsanity.com/paper/2406.11065/) (2024)
- [Automatic Prosody Annotation with Pre-Trained Text-Speech Model](https://mlsanity.com/paper/2206.07956/) (2022)
- [What does BERT learn about prosody?](https://mlsanity.com/paper/2304.12706/) (2023)
- [Investigating the Utility of Surprisal from Large Language Models for Speech Synthesis Prosody](https://mlsanity.com/paper/2306.09814/) (2023)
- [The time scale of redundancy between prosody and linguistic context](https://mlsanity.com/paper/2503.11630/) (2025)
- [DPP-TTS: Diversifying prosodic features of speech via determinantal point processes](https://mlsanity.com/paper/2310.14663/) (2023)
- [Spontaneous Speech Variables for Evaluating LLMs Cognitive Plausibility](https://mlsanity.com/paper/2505.16277/) (2025)
- [Word meaning co-determines vowel-inherent spectral change. A corpus-based investigation of conversational Mandarin](https://mlsanity.com/paper/2607.21391/) (2026)

## Abstract (arXiv)

In this paper we introduce a new natural language processing dataset and benchmark for predicting prosodic prominence from written text. To our knowledge this will be the largest publicly available dataset with prosodic labels. We describe the dataset construction and the resulting benchmark dataset in detail and train a number of different models ranging from feature-based classifiers to neural network systems for the prediction of discretized prosodic prominence. We show that pre-trained contextualized word representations from BERT outperform the other models even with less than 10% of the training data. Finally we discuss the dataset in light of the results and point to future research and plans for further improving both the dataset and methods of predicting prosodic prominence from text. The dataset and the code for the models are publicly available.