# Investigating the Utility of Surprisal from Large Language Models for Speech Synthesis Prosody

arXiv: https://arxiv.org/abs/2306.09814 · HTML: https://mlsanity.com/paper/2306.09814/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 16 Jun 2023.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [FastPitch: Parallel Text-to-speech with Pitch Prediction](https://mlsanity.com/paper/2006.06873/) (2020)
- [Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?](https://mlsanity.com/paper/2212.12131/) (2022)
- [One TTS Alignment To Rule Them All](https://mlsanity.com/paper/2108.10447/) (2021)
- [Predicting Prosodic Prominence from Text with Pre-trained Contextualized Word Representations](https://mlsanity.com/paper/1908.02262/) (2019)
- [Prosodic Prominence and Boundaries in Sequence-to-Sequence Speech Synthesis](https://mlsanity.com/paper/2006.15967/) (2020)
- [What does BERT learn about prosody?](https://mlsanity.com/paper/2304.12706/) (2023)

## Built on by (most cited first, 4 of 4)

- [ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering](https://mlsanity.com/paper/2401.07333/) (2024)
- [LLMs Meet Multimodal Generation and Editing: A Survey](https://mlsanity.com/paper/2405.19334/) (2024)
- [Investigating the Impact of Word Informativeness on Speech Emotion Recognition](https://mlsanity.com/paper/2506.02239/) (2025)
- [LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication](https://mlsanity.com/paper/2609.23193/) (2026)

## Abstract (arXiv)

This paper investigates the use of word surprisal, a measure of the predictability of a word in a given context, as a feature to aid speech synthesis prosody. We explore how word surprisal extracted from large language models (LLMs) correlates with word prominence, a signal-based measure of the salience of a word in a given discourse. We also examine how context length and LLM size affect the results, and how a speech synthesizer conditioned with surprisal values compares with a baseline system. To evaluate these factors, we conducted experiments using a large corpus of English text and LLMs of varying sizes. Our results show that word surprisal and word prominence are moderately correlated, suggesting that they capture related but distinct aspects of language use. We find that length of context and size of the LLM impact the correlations, but not in the direction anticipated, with longer contexts and larger LLMs generally underpredicting prominent words in a nearly linear manner. We demonstrate that, in line with these findings, a speech synthesizer conditioned with surprisal values provides a minimal improvement over the baseline with the results suggesting a limited effect of using surprisal values for eliciting appropriate prominence patterns.