mlsanityData to 14 Oct 2026

Simple and Effective Multi-sentence TTS with Expressive and Coherent Prosody

arXiv 2206.14643 ↗First posted 29 Jun 2022eess.AS cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Generating expressive and contextually appropriate prosody remains a challenge for modern text-to-speech (TTS) systems. This is particularly evident for long, multi-sentence inputs. In this paper, we examine simple extensions to a Transformer-based FastSpeech-like system, with the goal of improving prosody for multi-sentence TTS. We find that long context, powerful text features, and training on multi-speaker data all improve prosody. More interestingly, they result in synergies. Long context disambiguates prosody, improves coherence, and plays to the strengths of Transformers. Fine-tuning word-level features from a powerful language model, such as BERT, appears to profit from more training data, readily available in a multi-speaker setting. We look into objective metrics on pausing and pacing and perform thorough subjective evaluations for speech naturalness. Our main system, which incorporates all the extensions, achieves consistently strong results, including statistically significant improvements in speech naturalness over all its competitors.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 21 of its 33 references that are in the map. First posted on arXiv 29 Jun 2022.

Builds on

21 of 33 references.

References · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  4. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  5. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  6. 2017Generalized End-to-End Loss for Speaker Verificationcited by 293
  7. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  8. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  9. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  10. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  11. 2020MultiSpeech: Multi-Speaker Text to Speech with Transformercited by 42
  12. 2019DurIAN: Duration Informed Attention Network For Multimodal Synthesiscited by 33
9 more
  1. 2020Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesiscited by 27
  2. 2021Universal Neural Vocoding with Parallel WaveNetcited by 25
  3. 2019Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Modelscited by 10
  4. 2019Evaluating Long-form Text-to-Speech: Comparing the Ratings of Sentences and Paragraphscited by 8
  5. 2019In Other News: A Bi-style Text-to-speech Model for Synthesizing Newscaster Voice with Limited Datacited by 7
  6. 2022CopyCat2: A Single Model for Multi-Speaker TTS and Many-to-Many Fine-Grained Prosody Transfercited by 7
  7. 2020s-Transformer: Segment-Transformer for Robust Neural Speech Synthesiscited by 5
  8. 2018An Empirical Analysis of the Correlation of Syntax and Prosodycited by 4
  9. 2020Using previous acoustic context to improve Text-to-Speech synthesiscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 0; 2025: 0; 2026: 1.

5 papers.

  1. 2023DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Trainingcited by 16
  2. 2023PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-based Prosody Modelingcited by 5
  3. 2023Controllable Emphasis with zero data for text-to-speechcited by 4
  4. 2023A Comparative Analysis of Pretrained Language Models for Text-to-Speechcited by 1
  5. 2026HaikuS2S: A Cascaded System For Responding In Versecited by 0