# FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis

arXiv: https://arxiv.org/abs/2106.15123 · HTML: https://mlsanity.com/paper/2106.15123/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 29 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron](https://mlsanity.com/paper/1803.09047/) (2018)
- [FastPitch: Parallel Text-to-speech with Pitch Prediction](https://mlsanity.com/paper/2006.06873/) (2020)
- [Hierarchical Generative Modeling for Controllable Speech Synthesis](https://mlsanity.com/paper/1810.07217/) (2018)
- [Learning latent representations for style control and transfer in end-to-end speech synthesis](https://mlsanity.com/paper/1812.04342/) (2018)
- [Parallel Tacotron: Non-Autoregressive and Controllable TTS](https://mlsanity.com/paper/2010.11439/) (2020)
- [Robust and fine-grained prosody control of end-to-end speech synthesis](https://mlsanity.com/paper/1811.02122/) (2018)
- [Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis](https://mlsanity.com/paper/2002.03785/) (2020)
- [Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior](https://mlsanity.com/paper/2002.03788/) (2020)
- [VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Network](https://mlsanity.com/paper/2007.15256/) (2020)
- [Neural source-filter-based waveform model for statistical parametric speech synthesis](https://mlsanity.com/paper/1810.11946/) (2018)
- [Adversarially Trained End-to-end Korean Singing Voice Synthesis System](https://mlsanity.com/paper/1908.01919/) (2019)
- [Disentangling Timbre and Singing Style with Multi-singer Singing Synthesis System](https://mlsanity.com/paper/1910.13069/) (2019)
- [Knowledge-and-Data-Driven Amplitude Spectrum Prediction for Hierarchical Neural Vocoders](https://mlsanity.com/paper/2004.07832/) (2020)

## Built on by (most cited first, 12 of 12)

- [PromptTTS: Controllable Text-to-Speech with Text Descriptions](https://mlsanity.com/paper/2211.12171/) (2022)
- [TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models](https://mlsanity.com/paper/2308.14430/) (2023)
- [PromptTTS 2: Describing and Generating Voices with Text Prompt](https://mlsanity.com/paper/2309.02285/) (2023)
- [Heterogeneous Target Speech Separation](https://mlsanity.com/paper/2204.03594/) (2022)
- [PITS: Variational Pitch Inference without Fundamental Frequency for End-to-End Pitch-controllable TTS](https://mlsanity.com/paper/2302.12391/) (2023)
- [Controllable speech synthesis by learning discrete phoneme-level prosodic representations](https://mlsanity.com/paper/2211.16307/) (2022)
- [Controllable Accented Text-to-Speech Synthesis](https://mlsanity.com/paper/2209.10804/) (2022)
- [Enhancement of Pitch Controllability using Timbre-Preserving Pitch Augmentation in FastPitch](https://mlsanity.com/paper/2204.05753/) (2022)
- [MultiVerse: Efficient and Expressive Zero-Shot Multi-Task Text-to-Speech](https://mlsanity.com/paper/2410.03192/) (2024)
- [RESOUND: Speech Reconstruction from Silent Videos via Acoustic-Semantic Decomposed Modeling](https://mlsanity.com/paper/2505.22024/) (2025)
- [SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech](https://mlsanity.com/paper/2206.12132/) (2022)
- [Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation](https://mlsanity.com/paper/2409.09311/) (2024)

## Abstract (arXiv)

Methods for modeling and controlling prosody with acoustic features have been proposed for neural text-to-speech (TTS) models. Prosodic speech can be generated by conditioning acoustic features. However, synthesized speech with a large pitch-shift scale suffers from audio quality degradation, and speaker characteristics deformation. To address this problem, we propose a feed-forward Transformer based TTS model that is designed based on the source-filter theory. This model, called FastPitchFormant, has a unique structure that handles text and acoustic features in parallel. With modeling each feature separately, the tendency that the model learns the relationship between two features can be mitigated.