# BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model

arXiv: https://arxiv.org/abs/2207.01718 · HTML: https://mlsanity.com/paper/2207.01718/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 29 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesis](https://mlsanity.com/paper/1808.01410/) (2018)
- [fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit](https://mlsanity.com/paper/2109.06912/) (2021)
- [Controllable neural text-to-speech synthesis using intuitive prosodic features](https://mlsanity.com/paper/2009.06775/) (2020)
- [Predicting Prosodic Prominence from Text with Pre-trained Contextualized Word Representations](https://mlsanity.com/paper/1908.02262/) (2019)
- [CAMP: a Two-Stage Approach to Modelling Prosody in Context](https://mlsanity.com/paper/2011.01175/) (2020)
- [Prosodic Prominence and Boundaries in Sequence-to-Sequence Speech Synthesis](https://mlsanity.com/paper/2006.15967/) (2020)

## Built on by (most cited first, 10 of 10)

- [DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training](https://mlsanity.com/paper/2307.16549/) (2023)
- [WHISTRESS: Enriching Transcriptions with Sentence Stress Detection](https://mlsanity.com/paper/2505.19103/) (2025)
- [StressTest: Can YOUR Speech LM Handle the Stress?](https://mlsanity.com/paper/2505.22765/) (2025)
- [Can LLMs Understand the Implication of Emphasized Sentences in Dialogue?](https://mlsanity.com/paper/2406.11065/) (2024)
- [What does BERT learn about prosody?](https://mlsanity.com/paper/2304.12706/) (2023)
- [EE-TTS: Emphatic Expressive TTS with Linguistic Information](https://mlsanity.com/paper/2305.12107/) (2023)
- [Multi-Modal Automatic Prosody Annotation with Contrastive Pretraining of SSWP](https://mlsanity.com/paper/2309.05423/) (2023)
- [Improving French Synthetic Speech Quality via SSML Prosody Control](https://mlsanity.com/paper/2508.17494/) (2025)
- [Towards Joint Modeling of Dialogue Response and Speech Synthesis based on Large Language Model](https://mlsanity.com/paper/2309.11000/) (2023)
- [Word-wise intonation model for cross-language TTS systems](https://mlsanity.com/paper/2409.20374/) (2024)

## Abstract (arXiv)

Several recent studies have tested the use of transformer language model representations to infer prosodic features for text-to-speech synthesis (TTS). While these studies have explored prosody in general, in this work, we look specifically at the prediction of contrastive focus on personal pronouns. This is a particularly challenging task as it often requires semantic, discursive and/or pragmatic knowledge to predict correctly. We collect a corpus of utterances containing contrastive focus and we evaluate the accuracy of a BERT model, finetuned to predict quantized acoustic prominence features, on these samples. We also investigate how past utterances can provide relevant information for this prediction. Furthermore, we evaluate the controllability of pronoun prominence in a TTS model conditioned on acoustic prominence features.