mlsanityData to 14 Oct 2026

DiFlow-TTS: Compact and Low-Latency Zero-Shot Text-to-Speech with Discrete Flow Matching

arXiv 2509.09631 ↗First posted 11 Sep 2025cs.SD cs.CL cs.CVcited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Zero-shot text-to-speech (TTS) has made significant progress in replicating unseen voices, yet balancing generation quality and inference efficiency remains challenging. Autoregressive models suffer from high latency, while diffusion-based approaches are constrained by training-time configurations. Moreover, most flow-based methods operate in continuous space, which introduces optimization challenges because continuous token spaces are inherently more complex than discrete ones. To address these limitations, we propose DiFlow-TTS, a novel zero-shot TTS framework based on discrete flow matching. The model consists of a deterministic Phoneme-Content Mapper for linguistic modeling and a Factorized Discrete Flow Denoiser that simultaneously generates prosody and acoustic token streams. Experimental results demonstrate the effectiveness of our approach across multiple evaluation metrics.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 49 of its 54 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

49 of 54 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching19 May
  2. 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation6 Feb
  3. 2024StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching6 Dec
  4. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  5. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors17 Jun
  6. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models5 Mar
  7. 2023ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speech29 Sep
  8. 2023DCTTS: Discrete Diffusion Model with Contrastive Learning for Text-to-speech Generation13 Sep
  9. 2023Matcha-TTS: A fast TTS architecture with conditional flow matching6 Sep
  10. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr
  11. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generation20 Jul

Other references · 38

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  9. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  10. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  11. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  12. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
26 more
  1. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  2. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  3. 2022High Fidelity Neural Audio Compressioncited by 687
  4. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  5. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  6. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  7. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  8. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  9. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 340
  10. 2024Discrete Flow Matchingcited by 317
  11. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  12. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  13. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  14. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 120
  15. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  16. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 110
  17. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 92
  18. 2024DeFoG: Discrete Flow Matching for Graph Generationcited by 82
  19. 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65
  20. 2023TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Modelscited by 48
  21. 2023UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocodingcited by 37
  22. 2023Amphion: An Open-Source Audio, Music and Speech Generation Toolkitcited by 27
  23. 2025MaskFlow: Discrete Flows For Flexible and Efficient Long Video Generationcited by 12
  24. 2025Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Modelcited by 8
  25. 2025All-atom inverse protein folding through discrete flow matchingcited by 8
  26. 2025Rethinking Discrete Tokens: Treating Them as Conditions for Continuous Autoregressive Image Synthesiscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Flamed-TTS: Flow Matching Attention-Free Models for Efficient Generating and Dynamic Pacing Zero-shot Text-to-Speech
  2. 2024An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS
  3. 2025Towards Lightweight and Stable Zero-shot TTS with Self-distilled Representation Disentanglement
  4. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching
  5. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS
  6. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  7. 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speech
  8. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching