# Embedding a Differentiable Mel-cepstral Synthesis Filter to a Neural Speech Synthesis System

arXiv: https://arxiv.org/abs/2211.11222 · HTML: https://mlsanity.com/paper/2211.11222/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [Neural source-filter waveform models for statistical parametric speech synthesis](https://mlsanity.com/paper/1904.12088/) (2019)
- [Sinsy: A Deep Neural Network-Based Singing Voice Synthesis System](https://mlsanity.com/paper/2108.02776/) (2021)
- [Speech Synthesis and Control Using Differentiable DSP](https://mlsanity.com/paper/2010.15084/) (2020)
- [Fast and High-Quality Singing Voice Synthesis System based on Convolutional Neural Networks](https://mlsanity.com/paper/1910.11690/) (2019)
- [Quasi-Periodic WaveNet: An Autoregressive Raw Waveform Generative Model with Pitch-dependent Dilated Convolution Neural Network](https://mlsanity.com/paper/2007.05663/) (2020)
- [Differentiable WORLD Synthesizer-based Neural Vocoder With Application To End-To-End Audio Style Transfer](https://mlsanity.com/paper/2208.07282/) (2022)
- [ExcitNet vocoder: A neural excitation model for parametric speech synthesis systems](https://mlsanity.com/paper/1811.04769/) (2018)

## Built on by (most cited first, 1 of 1)

- [Fast, High-Quality and Parameter-Efficient Articulatory Synthesis using Differentiable DSP](https://mlsanity.com/paper/2409.02451/) (2024)

## Abstract (arXiv)

This paper integrates a classic mel-cepstral synthesis filter into a modern neural speech synthesis system towards end-to-end controllable speech synthesis. Since the mel-cepstral synthesis filter is explicitly embedded in neural waveform models in the proposed system, both voice characteristics and the pitch of synthesized speech are highly controlled via a frequency warping parameter and fundamental frequency, respectively. We implement the mel-cepstral synthesis filter as a differentiable and GPU-friendly module to enable the acoustic and waveform models in the proposed system to be simultaneously optimized in an end-to-end manner. Experiments show that the proposed system improves speech quality from a baseline system maintaining controllability. The core PyTorch modules used in the experiments will be publicly available on GitHub.