# ComedicSpeech: Text To Speech For Stand-up Comedies in Low-Resource Scenarios

arXiv: https://arxiv.org/abs/2305.12200 · HTML: https://mlsanity.com/paper/2305.12200/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 21 references that are in the map.
- First posted on arXiv 20 May 2023.

## Builds on (most cited first, 16 of 16)

- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Hybrid Transformers for Music Source Separation](https://mlsanity.com/paper/2211.08553/) (2022)
- [AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines](https://mlsanity.com/paper/2010.11567/) (2020)
- [AdaSpeech: Adaptive Text to Speech for Custom Voice](https://mlsanity.com/paper/2103.00993/) (2021)
- [Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation](https://mlsanity.com/paper/2106.03153/) (2021)
- [Hybrid Spectrogram and Waveform Source Separation](https://mlsanity.com/paper/2111.03600/) (2021)
- [DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speech](https://mlsanity.com/paper/2207.01063/) (2022)
- [AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarios](https://mlsanity.com/paper/2204.00436/) (2022)
- [Conversational End-to-End TTS for Voice Agent](https://mlsanity.com/paper/2005.10438/) (2020)
- [Controllable Context-aware Conversational Speech Synthesis](https://mlsanity.com/paper/2106.10828/) (2021)
- [AdaSpeech 3: Adaptive Text to Speech for Spontaneous Style](https://mlsanity.com/paper/2107.02530/) (2021)
- [Content-Dependent Fine-Grained Speaker Embedding for Zero-Shot Speaker Adaptation in Text-to-Speech Synthesis](https://mlsanity.com/paper/2204.00990/) (2022)
- [Self-supervised Context-aware Style Representation for Expressive Speech Synthesis](https://mlsanity.com/paper/2206.12559/) (2022)

## Abstract (arXiv)

Text to Speech (TTS) models can generate natural and high-quality speech, but it is not expressive enough when synthesizing speech with dramatic expressiveness, such as stand-up comedies. Considering comedians have diverse personal speech styles, including personal prosody, rhythm, and fillers, it requires real-world datasets and strong speech style modeling capabilities, which brings challenges. In this paper, we construct a new dataset and develop ComedicSpeech, a TTS system tailored for the stand-up comedy synthesis in low-resource scenarios. First, we extract prosody representation by the prosody encoder and condition it to the TTS model in a flexible way. Second, we enhance the personal rhythm modeling by a conditional duration predictor. Third, we model the personal fillers by introducing comedian-related special tokens. Experiments show that ComedicSpeech achieves better expressiveness than baselines with only ten-minute training data for each comedian. The audio samples are available at https://xh621.github.io/stand-up-comedy-demo/