# Differentiable WORLD Synthesizer-based Neural Vocoder With Application To End-To-End Audio Style Transfer

arXiv: https://arxiv.org/abs/2208.07282 · HTML: https://mlsanity.com/paper/2208.07282/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 12 of 12)

- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [Real-time Speech Frequency Bandwidth Extension](https://mlsanity.com/paper/2010.10677/) (2020)
- [A Neural Parametric Singing Synthesizer](https://mlsanity.com/paper/1704.03809/) (2017)
- [WGANSing: A Multi-Voice Singing Voice Synthesizer Based on the Wasserstein-GAN](https://mlsanity.com/paper/1903.10729/) (2019)
- [Speech Synthesis and Control Using Differentiable DSP](https://mlsanity.com/paper/2010.15084/) (2020)
- [HooliGAN: Robust, High Quality Neural Vocoding](https://mlsanity.com/paper/2008.02493/) (2020)
- [Using Cyclic Noise as the Source Signal for Neural Source-Filter-based Speech Waveform Model](https://mlsanity.com/paper/2004.02191/) (2020)

## Built on by (most cited first, 3 of 3)

- [InstrumentGen: Generating Sample-Based Musical Instruments From Text](https://mlsanity.com/paper/2311.04339/) (2023)
- [Fast, High-Quality and Parameter-Efficient Articulatory Synthesis using Differentiable DSP](https://mlsanity.com/paper/2409.02451/) (2024)
- [Embedding a Differentiable Mel-cepstral Synthesis Filter to a Neural Speech Synthesis System](https://mlsanity.com/paper/2211.11222/) (2022)

## Abstract (arXiv)

In this paper, we propose a differentiable WORLD synthesizer and demonstrate its use in end-to-end audio style transfer tasks such as (singing) voice conversion and the DDSP timbre transfer task. Accordingly, our baseline differentiable synthesizer has no model parameters, yet it yields adequate synthesis quality. We can extend the baseline synthesizer by appending lightweight black-box postnets which apply further processing to the baseline output in order to improve fidelity. An alternative differentiable approach considers extraction of the source excitation spectrum directly, which can improve naturalness albeit for a narrower class of style transfer applications. The acoustic feature parameterization used by our approaches has the added benefit that it naturally disentangles pitch and timbral information so that they can be modeled separately. Moreover, as there exists a robust means of estimating these acoustic features from monophonic audio sources, it allows for parameter loss terms to be added to an end-to-end objective function, which can help convergence and/or further stabilize (adversarial) training.