Improving the expressiveness of neural vocoding with non-affine Normalizing Flows
Not on the map. Why?
Abstract (arXiv)
This paper proposes a general enhancement to the Normalizing Flows (NF) used in neural vocoding. As a case study, we improve expressive speech vocoding with a revamped Parallel Wavenet (PW). Specifically, we propose to extend the affine transformation of PW to the more expressive invertible non-affine function. The greater expressiveness of the improved PW leads to better-perceived signal quality and naturalness in the waveform reconstruction and text-to-speech (TTS) tasks. We evaluate the model across different speaking styles on a multi-speaker, multi-lingual dataset. In the waveform reconstruction task, the proposed model closes the naturalness and signal quality gap from the original PW to recordings by 10%, and from other state-of-the-art neural vocoding systems by more than 60%. We also demonstrate improvements in objective metrics on the evaluation test set with L2 Spectral Distance and Cross-Entropy reduced by 3% and 6unicodex2030 comparing to the affine PW. Furthermore, we extend the probability density distillation procedure proposed by the original PW paper, so that it works with any non-affine invertible and differentiable function.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 16 Jun 2021.
Builds on
32 of 40 references.
References · 32
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2013Auto-Encoding Variational Bayescited by 3,980
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
- 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2016Conditional Image Generation with PixelCNN Decoderscited by 859
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
- 2019Neural Spline Flowscited by 590
- 2018Invertible Residual Networkscited by 364
20 more
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
- 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
- 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
- 2019Residual Flows for Invertible Generative Modelingcited by 256
- 2018Efficient Neural Audio Synthesiscited by 218
- 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
- 2018Neural Autoregressive Flowscited by 207
- 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
- 2018Neural Importance Samplingcited by 154
- 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
- 2018ClariNet: Parallel Wave Generation in End-to-End Text-to-Speechcited by 70
- 2019Block Neural Autoregressive Flowcited by 62
- 2019Sum-of-Squares Polynomial Flowcited by 62
- 2018FloWaveNet : A Generative Flow for Raw Audiocited by 54
- 2019MelNet: A Generative Model for Audio in the Frequency Domaincited by 41
- 2020StyleMelGAN: An Efficient High-Fidelity Adversarial Vocoder with Temporal Adaptive Normalizationcited by 26
- 2021Universal Neural Vocoding with Parallel WaveNetcited by 25
- 2019Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protectioncited by 19
- 2020WaveNODE: A Continuous Normalizing Flow for Speech Synthesiscited by 5
Built on by
Not cited yet.