mlsanityData to 7 Oct 2026

Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input

arXiv 2102.09914 ↗First posted 19 Feb 2021cs.CL eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

The prosody of a spoken word is determined by its surrounding context. In incremental text-to-speech synthesis, where the synthesizer produces an output before it has access to the complete input, the full context is often unknown which can result in a loss of naturalness in the synthesized speech. In this paper, we investigate whether the use of predicted future text can attenuate this loss. We compare several test conditions of next future word: (a) unknown (zero-word), (b) language model predicted, (c) randomly predicted and (d) ground-truth. We measure the prosodic features (pitch, energy and duration) and find that predicted text provides significant improvements over a zero-word lookahead, but only slight gains over random-word lookahead. We confirm these results with a perceptive test.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 19 Feb 2021.

Builds on

16 of 33 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  4. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  5. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  6. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
  7. 2018Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesiscited by 30
  8. 2019Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Frameworkcited by 15
  9. 2020Simultaneous Speech-to-Speech Translation System with Neural Incremental ASR, MT, and TTScited by 12
  10. 2019Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Modelscited by 10
  11. 2019Speculative Beam Search for Simultaneous Translationcited by 10
  12. 2020What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTScited by 9
4 more
  1. 2020Incremental Text to Speech for Neural Sequence-to-Sequence Models using Reinforcement Learningcited by 8
  2. 2020Incremental Text-to-Speech Synthesis Using Pseudo Lookahead with Large Pretrained Language Modelcited by 7
  3. 2019Learning Dynamic Author Representations with Temporal Language Modelscited by 5
  4. 2020Incremental Machine Speech Chain Towards Enabling Listening while Speaking in Real-timecited by 1

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 1; 2023: 0; 2024: 1.

7 papers.

  1. 2021A Survey on Neural Speech Synthesiscited by 182
  2. 2021Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attentioncited by 9
  3. 2021Review of end-to-end speech synthesis technology based on deep learningcited by 6
  4. 2021From Start to Finish: Latency Reduction Strategies for Incremental Speech Synthesis in Simultaneous Speech-to-Speech Translationcited by 3
  5. 2021Low-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Networkcited by 2
  6. 2022Efficient Incremental Text-to-Speech on GPUscited by 1
  7. 2024Llama-VITS: Enhancing TTS Synthesis with Semantic Awarenesscited by 1