Beyond Two-stage Diffusion TTS: Joint Structure and Content Refinement via Jump Diffusion
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
Diffusion and flow matching TTS faces a tension between discrete temporal structure and continuous spectral modeling. Two-stage models diffuse on fixed alignments, often collapsing to mean prosody; single-stage models avoid explicit durations but suffer alignment instability. We propose a jump-diffusion framework where discrete jumps model temporal structure and continuous diffusion refines spectral content within one process. Even in its one-shot degenerate form, our framework achieves 3.37% WER vs. 4.38% for Grad-TTS with improved UTMOSv2 on LJSpeech. The full iterative UDD variant further enables adaptive prosody, autonomously inserting natural pauses in out-of-distribution slow speech rather than stretching uniformly. Audio samples are available at https://anonymousinterpseech.github.io/TTS_Demo/.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 23 of its 26 references that are in the map. First posted on arXiv 14 Mar 2026.
Builds on
23 of 26 references.
Closest ancestors · 6
Same direction, newest first
- 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesis4 Dec
- 2025Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling26 May
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
- 2023E3 TTS: Easy End-to-End Diffusion-based Text to Speech2 Nov
- 2023Matcha-TTS: A fast TTS architecture with conditional flow matching6 Sep
- 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May
Other references · 17
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
- 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
- 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
- 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 149
- 2024Generator Matching: Generative modeling with arbitrary Markov processescited by 82
5 more
- 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 78
- 2023Trans-Dimensional Generative Modeling via Jump Diffusion Modelscited by 33
- 2024Heavy-Tailed Diffusion with Denoising L'evy Probabilistic Modelscited by 16
- 2024Should you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speechcited by 5
- 2024Piecewise deterministic generative modelscited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Stage-adaptive audio diffusion modeling
- 2025Posterior Transition Modeling for Unsupervised Diffusion-Based Speech Enhancement
- 2026DiTAR+: Dual Optimization for Robust Autoregressive Diffusion Speech Synthesis
- 2026Harmonizing Spectral Evolution in Conditional Flow Matching for TTS
- 2026Combining Masked Language Modeling and Cross-Modal Contrastive Learning for Prosody-Aware TTS ↗
- 2024Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation
- 2023Duplex Diffusion Models Improve Speech-to-Speech Translation
- 2024On the Semantic Latent Space of Diffusion-Based Text-to-Speech Models