mlsanityData to 14 Oct 2026

Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis

arXiv 2507.00227 ↗First posted 30 Jun 2025eess.AS cs.AIcited by 2

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

While generative methods have progressed rapidly in recent years, generating expressive prosody for an utterance remains a challenging task in text-to-speech synthesis. This is particularly true for systems that model prosody explicitly through parameters such as pitch, energy, and duration, which is commonly done for the sake of interpretability and controllability. In this work, we investigate the effectiveness of stochastic methods for this task, including Normalizing Flows, Conditional Flow Matching, and Rectified Flows. We compare these methods to a traditional deterministic baseline, as well as to real human realizations. Our extensive subjective and objective evaluations demonstrate that stochastic methods produce natural prosody on par with human speakers by capturing the variability inherent in human speech. Further, they open up additional controllability options by allowing the sampling temperature to be tuned.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 29 of its 36 references that are in the map. First posted on arXiv 30 Jun 2025.

Builds on

29 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Matcha-TTS: A fast TTS architecture with conditional flow matching6 Sep
  2. 2023Better speech synthesis through scaling12 May
  3. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr

Other references · 26

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  4. 2015Variational Inference with Normalizing Flowscited by 2,170
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  6. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  8. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  9. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  10. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  11. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  12. 2022Rectified Flow: A Marginal Preserving Approach to Optimal Transportcited by 283
14 more
  1. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
  2. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  3. 2025ASVspoof 5: Design, Collection and Validation of Resources for Spoofing, Deepfake, and Adversarial Attack Detection Using Crowdsourced Speechcited by 26
  4. 2023A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speechcited by 22
  5. 2022Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Featurescited by 20
  6. 2021DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021cited by 19
  7. 2022DelightfulTTS 2: End-to-End Speech Synthesis with Adversarial Vector-Quantized Auto-Encoderscited by 15
  8. 2022Exact Prosody Cloning in Zero-Shot Multispeaker Text-to-Speechcited by 13
  9. 2024Meta Learning Text-to-Speech Synthesis in over 7000 Languagescited by 9
  10. 2023The IMS Toucan System for the Blizzard Challenge 2023cited by 6
  11. 2022PoeticTTS -- Controllable Poetry Reading for Literary Studiescited by 5
  12. 2024Should you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speechcited by 5
  13. 2021ADEPT: A Dataset for Evaluating Prosody Transfercited by 4
  14. 2023Varianceflow: High-Quality and Controllable Text-to-Speech using Variance Information via Normalizing Flowcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Modelscited by 0
  2. 2026Mask, Sample, Revise: A Revisable CTMC Inference Stack for Guided Discrete Flow Matching Text-to-Speechcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Expressive Zero-Shot Speech Synthesis with Hierarchical Prosody Modeling
  2. 2023Diverse and Expressive Speech Prosody Prediction with Denoising Diffusion Probabilistic Model
  3. 2021On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis
  4. 2023Comparing normalizing flows and diffusion models for prosody and acoustic modelling in text-to-speech
  5. 2023DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training
  6. 2025DrawSpeech: Expressive Speech Synthesis Using Prosodic Sketches as Control Conditions
  7. 2024ProsodyFM: Unsupervised Phrasing and Intonation Control for Intelligible Speech Synthesis
  8. 2026Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis