# Revisiting Over-Smoothness in Text to Speech

arXiv: https://arxiv.org/abs/2202.13066 · HTML: https://mlsanity.com/paper/2202.13066/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2022.

## Built on by (most cited first, 30 of 32)

- [ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech](https://mlsanity.com/paper/2207.06389/) (2022)
- [BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data](https://mlsanity.com/paper/2402.08093/) (2024)
- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Models](https://mlsanity.com/paper/2211.09383/) (2022)
- [ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching](https://mlsanity.com/paper/2506.13053/) (2025)
- [Towards Building Text-To-Speech Systems for the Next Billion Users](https://mlsanity.com/paper/2211.09536/) (2022)
- [FlashSpeech: Efficient Zero-Shot Speech Synthesis](https://mlsanity.com/paper/2404.14700/) (2024)
- [FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models](https://mlsanity.com/paper/2305.13612/) (2023)
- [DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training](https://mlsanity.com/paper/2307.16549/) (2023)
- [RMSSinger: Realistic-Music-Score based Singing Voice Synthesis](https://mlsanity.com/paper/2305.10686/) (2023)
- [Let There Be Sound: Reconstructing High Quality Speech from Silent Videos](https://mlsanity.com/paper/2308.15256/) (2023)
- [FELLE: Autoregressive Speech Synthesis with Token-Wise Coarse-to-Fine Flow Matching](https://mlsanity.com/paper/2502.11128/) (2025)
- [Takin: A Cohort of Superior Quality Zero-shot Speech Generation Models](https://mlsanity.com/paper/2409.12139/) (2024)
- [FluentEditor: Text-based Speech Editing by Considering Acoustic and Prosody Consistency](https://mlsanity.com/paper/2309.11725/) (2023)
- [PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-based Prosody Modeling](https://mlsanity.com/paper/2306.07489/) (2023)
- [Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis](https://mlsanity.com/paper/2404.19622/) (2024)
- [Diverse and Expressive Speech Prosody Prediction with Denoising Diffusion Probabilistic Model](https://mlsanity.com/paper/2305.16749/) (2023)
- [NaturalL2S: End-to-End High-quality Multispeaker Lip-to-Speech Synthesis with Differential Digital Signal Processing](https://mlsanity.com/paper/2502.12002/) (2025)
- [Multilingual Multiaccented Multispeaker TTS with RADTTS](https://mlsanity.com/paper/2301.10335/) (2023)
- [AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment](https://mlsanity.com/paper/2305.04476/) (2023)
- [Towards Robust FastSpeech 2 by Modelling Residual Multimodality](https://mlsanity.com/paper/2306.01442/) (2023)
- [Multi-modal Adversarial Training for Zero-Shot Voice Cloning](https://mlsanity.com/paper/2408.15916/) (2024)
- [Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters](https://mlsanity.com/paper/2507.04817/) (2025)
- [Acquiring Pronunciation Knowledge from Transcribed Speech Audio via Multi-task Learning](https://mlsanity.com/paper/2409.09891/) (2024)
- [Lina-Speech: Gated Linear Attention and Initial-State Tuning for Multi-Sample Prompting Text-To-Speech Synthesis](https://mlsanity.com/paper/2410.23320/) (2024)
- [SCRAPS: Speech Contrastive Representations of Acoustic and Phonetic Spaces](https://mlsanity.com/paper/2307.12445/) (2023)
- [Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation](https://mlsanity.com/paper/2409.09311/) (2024)
- [BemaGANv2: Discriminator Combination Strategies for GAN-based Vocoders in Long-Term Audio Generation](https://mlsanity.com/paper/2506.09487/) (2025)
- [A survey of AI-generated voices and their detection](https://mlsanity.com/paper/2608.15411/) (2026)
- [Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models](https://mlsanity.com/paper/2609.01723/) (2026)

## Abstract (arXiv)

Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast generation speed. One limitation of NAR-TTS models is that they ignore the correlation in time and frequency domains while generating speech mel-spectrograms, and thus cause blurry and over-smoothed results. In this work, we revisit this over-smoothing problem from a novel perspective: the degree of over-smoothness is determined by the gap between the complexity of data distributions and the capability of modeling methods. Both simplifying data distributions and improving modeling methods can alleviate the problem. Accordingly, we first study methods reducing the complexity of data distributions. Then we conduct a comprehensive study on NAR-TTS models that use some advanced modeling methods. Based on these studies, we find that 1) methods that provide additional condition inputs reduce the complexity of data distributions to model, thus alleviating the over-smoothing problem and achieving better voice quality. 2) Among advanced modeling methods, Laplacian mixture loss performs well at modeling multimodal distributions and enjoys its simplicity, while GAN and Glow achieve the best voice quality while suffering from increased training or model complexity. 3) The two categories of methods can be combined to further alleviate the over-smoothness and improve the voice quality. 4) Our experiments on the multi-speaker dataset lead to similar conclusions as above and providing more variance information can reduce the difficulty of modeling the target data distribution and alleviate the requirements for model capacity.