# Statistical Parametric Speech Synthesis Using Generative Adversarial Networks Under A Multi-task Learning Framework

arXiv: https://arxiv.org/abs/1707.01670 · HTML: https://mlsanity.com/paper/1707.01670/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2017.

## Built on by (most cited first, 5 of 5)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Audio representations for deep learning in sound synthesis: A review](https://mlsanity.com/paper/2201.02490/) (2022)
- [Towards Generating Large Synthetic Phytoplankton Datasets for Efficient Monitoring of Harmful Algal Blooms](https://mlsanity.com/paper/2208.02332/) (2022)
- [Machines & Influence: An Information Systems Lens](https://mlsanity.com/paper/2111.13365/) (2021)
- [Unifying conditional and unconditional semantic image synthesis with OCO-GAN](https://mlsanity.com/paper/2211.14105/) (2022)

## Abstract (arXiv)

In this paper, we aim at improving the performance of synthesized speech in statistical parametric speech synthesis (SPSS) based on a generative adversarial network (GAN). In particular, we propose a novel architecture combining the traditional acoustic loss function and the GAN's discriminative loss under a multi-task learning (MTL) framework. The mean squared error (MSE) is usually used to estimate the parameters of deep neural networks, which only considers the numerical difference between the raw audio and the synthesized one. To mitigate this problem, we introduce the GAN as a second task to determine if the input is a natural speech with specific conditions. In this MTL framework, the MSE optimization improves the stability of GAN, and at the same time GAN produces samples with a distribution closer to natural speech. Listening tests show that the multi-task architecture can generate more natural speech that satisfies human perception than the conventional methods.