ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
Not on the map. Why?
Abstract (arXiv)
While state-of-the-art Text-to-Speech systems can generate natural speech of very high quality at sentence level, they still meet great challenges in speech generation for paragraph / long-form reading. Such deficiencies are due to i) ignorance of cross-sentence contextual information, and ii) high computation and memory cost for long-form synthesis. To address these issues, this work develops a lightweight yet effective TTS system, ContextSpeech. Specifically, we first design a memory-cached recurrence mechanism to incorporate global text and speech context into sentence encoding. Then we construct hierarchically-structured textual semantics to broaden the scope for global context enhancement. Additionally, we integrate linearized self-attention to improve model efficiency. Experiments show that ContextSpeech significantly improves the voice quality and prosody expressiveness in paragraph reading with competitive model efficiency. Audio samples are available at: https://contextspeech.github.io/demo/
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 26 of its 28 references that are in the map. First posted on arXiv 3 Jul 2023.
Builds on
26 of 28 references.
References · 26
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
- 2020Reformer: The Efficient Transformercited by 1,668
- 2019Generating Long Sequences with Sparse Transformerscited by 1,546
- 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
14 more
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
- 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
- 2018Efficient Neural Audio Synthesiscited by 219
- 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 215
- 2021Conformer: Local Features Coupling Global Representations for Visual Recognitioncited by 164
- 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
- 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
- 2020Conversational End-to-End TTS for Voice Agentcited by 37
- 2020Improving Prosody Modelling with Cross-Utterance BERT Embeddings for End-to-end Speech Synthesiscited by 27
- 2021DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021cited by 19
- 2021PermuteFormer: Efficient Relative Position Encoding for Long Sequencescited by 14
- 2022ParaTTS: Learning Linguistic and Prosodic Cross-sentence Information in Paragraph-based TTScited by 5
- 2022Towards Expressive Speaking Style Modelling with Hierarchical Context Information for Mandarin Speech Synthesiscited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.