# A Neural Vocoder with Hierarchical Generation of Amplitude and Phase Spectra for Statistical Parametric Speech Synthesis

arXiv: https://arxiv.org/abs/1906.09573 · HTML: https://mlsanity.com/paper/1906.09573/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2019.

## Built on by (most cited first, 10 of 10)

- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [Visual Acoustic Matching](https://mlsanity.com/paper/2202.06875/) (2022)
- [CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing](https://mlsanity.com/paper/2202.09950/) (2022)
- [HiddenSinger: High-Quality Singing Voice Synthesis via Neural Audio Codec and Latent Diffusion Models](https://mlsanity.com/paper/2306.06814/) (2023)
- [Singing-Tacotron: Global duration control attention and dynamic filter for End-to-end singing voice synthesis](https://mlsanity.com/paper/2202.07907/) (2022)
- [Unified Source-Filter GAN: Unified Source-filter Network Based On Factorization of Quasi-Periodic Parallel WaveGAN](https://mlsanity.com/paper/2104.04668/) (2021)
- [NeuralDPS: Neural Deterministic Plus Stochastic Model with Multiband Excitation for Noise-Controllable Waveform Generation](https://mlsanity.com/paper/2203.02678/) (2022)
- [Stage-Wise and Prior-Aware Neural Speech Phase Prediction](https://mlsanity.com/paper/2410.04990/) (2024)

## Abstract (arXiv)

This paper presents a neural vocoder named HiNet which reconstructs speech waveforms from acoustic features by predicting amplitude and phase spectra hierarchically. Different from existing neural vocoders such as WaveNet, SampleRNN and WaveRNN which directly generate waveform samples using single neural networks, the HiNet vocoder is composed of an amplitude spectrum predictor (ASP) and a phase spectrum predictor (PSP). The ASP is a simple DNN model which predicts log amplitude spectra (LAS) from acoustic features. The predicted LAS are sent into the PSP for phase recovery. Considering the issue of phase warping and the difficulty of phase modeling, the PSP is constructed by concatenating a neural source-filter (NSF) waveform generator with a phase extractor. We also introduce generative adversarial networks (GANs) into both ASP and PSP. Finally, the outputs of ASP and PSP are combined to reconstruct speech waveforms by short-time Fourier synthesis. Since there are no autoregressive structures in both predictors, the HiNet vocoder can generate speech waveforms with high efficiency. Objective and subjective experimental results show that our proposed HiNet vocoder achieves better naturalness of reconstructed speech than the conventional STRAIGHT vocoder, a 16-bit WaveNet vocoder using open source implementation and an NSF vocoder with similar complexity to the PSP and obtains similar performance with a 16-bit WaveRNN vocoder. We also find that the performance of HiNet is insensitive to the complexity of the neural waveform generator in PSP to some extend. After simplifying its model structure, the time consumed for generating 1s waveforms of 16kHz speech using a GPU can be further reduced from 0.34s to 0.19s without significant quality degradation.