# Neural source-filter waveform models for statistical parametric speech synthesis

arXiv: https://arxiv.org/abs/1904.12088 · HTML: https://mlsanity.com/paper/1904.12088/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2019.

## Built on by (most cited first, 30 of 46)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [RAVE: A variational autoencoder for fast and high-quality neural audio synthesis](https://mlsanity.com/paper/2111.05011/) (2021)
- [SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shaping](https://mlsanity.com/paper/2203.16749/) (2022)
- [MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modeling](https://mlsanity.com/paper/2112.09312/) (2021)
- [The Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2306.14422/) (2023)
- [PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptions](https://mlsanity.com/paper/2309.08140/) (2023)
- [SpoofCeleb: Speech Deepfake Detection and SASV In The Wild](https://mlsanity.com/paper/2409.17285/) (2024)
- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)
- [Differentially Private Speaker Anonymization](https://mlsanity.com/paper/2202.11823/) (2022)
- [LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning](https://mlsanity.com/paper/2406.07969/) (2024)
- [DDX7: Differentiable FM Synthesis of Musical Instrument Sounds](https://mlsanity.com/paper/2208.06169/) (2022)
- [Neural Waveshaping Synthesis](https://mlsanity.com/paper/2107.05050/) (2021)
- [One Billion Audio Sounds from GPU-enabled Modular Synthesis](https://mlsanity.com/paper/2104.12922/) (2021)
- [Source-Filter HiFi-GAN: Fast and Pitch Controllable High-Fidelity Neural Vocoder](https://mlsanity.com/paper/2210.15533/) (2022)
- [HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transform](https://mlsanity.com/paper/2309.09493/) (2023)
- [Are disentangled representations all you need to build speaker anonymization systems?](https://mlsanity.com/paper/2208.10497/) (2022)
- [Audio representations for deep learning in sound synthesis: A review](https://mlsanity.com/paper/2201.02490/) (2022)
- [Music Separation Enhancement with Generative Modeling](https://mlsanity.com/paper/2208.12387/) (2022)
- [Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transport](https://mlsanity.com/paper/2312.14507/) (2023)
- [PeriodNet: A non-autoregressive waveform generation model with a structure separating periodic and aperiodic components](https://mlsanity.com/paper/2102.07786/) (2021)
- [Period VITS: Variational Inference with Explicit Pitch Modeling for End-to-end Emotional Speech Synthesis](https://mlsanity.com/paper/2210.15964/) (2022)
- [LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance](https://mlsanity.com/paper/2406.05325/) (2024)
- [CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGAN](https://mlsanity.com/paper/2104.00353/) (2021)
- [Decoupling Speaker-Independent Emotions for Voice Conversion Via Source-Filter Networks](https://mlsanity.com/paper/2110.01164/) (2021)
- [Differentiable Modal Synthesis for Physical Modeling of Planar String Sound and Motion Simulation](https://mlsanity.com/paper/2407.05516/) (2024)
- [Unified Source-Filter GAN: Unified Source-filter Network Based On Factorization of Quasi-Periodic Parallel WaveGAN](https://mlsanity.com/paper/2104.04668/) (2021)
- [LoopNet: Musical Loop Synthesis Conditioned On Intuitive Musical Parameters](https://mlsanity.com/paper/2105.10371/) (2021)
- [NNSVS: A Neural Network-Based Singing Voice Synthesis Toolkit](https://mlsanity.com/paper/2210.15987/) (2022)
- [SiFiSinger: A High-Fidelity End-to-End Singing Voice Synthesizer based on Source-filter Model](https://mlsanity.com/paper/2410.12536/) (2024)

## Abstract (arXiv)

Neural waveform models such as WaveNet have demonstrated better performance than conventional vocoders for statistical parametric speech synthesis. As an autoregressive (AR) model, WaveNet is limited by a slow sequential waveform generation process. Some new models that use the inverse-autoregressive flow (IAF) can generate a whole waveform in a one-shot manner. However, these IAF-based models require sequential transformation during training, which severely slows down the training speed. Other models such as Parallel WaveNet and ClariNet bring together the benefits of AR and IAF-based models and train an IAF model by transferring the knowledge from a pre-trained AR teacher to an IAF student without any sequential transformation. However, both models require additional training criteria, and their implementation is prohibitively complicated. We propose a framework for neural source-filter (NSF) waveform modeling without AR nor IAF-based approaches. This framework requires only three components for waveform generation: a source module that generates a sine-based signal as excitation, a non-AR dilated-convolution-based filter module that transforms the excitation into a waveform, and a conditional module that pre-processes the acoustic features for the source and filer modules. This framework minimizes spectral-amplitude distances for model training, which can be efficiently implemented by using short-time Fourier transform routines. Under this framework, we designed three NSF models and compared them with WaveNet. It was demonstrated that the NSF models generated waveforms at least 100 times faster than WaveNet, and the quality of the synthetic speech from the best NSF model was better than or equally good as that from WaveNet.