mlsanityData to 14 Oct 2026

VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching

arXiv 2309.05027 ↗First posted 10 Sep 2023eess.AS cs.AI cs.HCcited by 49

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Although diffusion models in text-to-speech have become a popular choice due to their strong generative ability, the intrinsic complexity of sampling from diffusion models harms their efficiency. Alternatively, we propose VoiceFlow, an acoustic model that utilizes a rectified flow matching algorithm to achieve high synthesis quality with a limited number of sampling steps. VoiceFlow formulates the process of generating mel-spectrograms into an ordinary differential equation conditional on text inputs, whose vector field is then estimated. The rectified flow technique then effectively straightens its sampling trajectory for efficient synthesis. Subjective and objective evaluations on both single and multi-speaker corpora showed the superior synthesis quality of VoiceFlow compared to the diffusion counterpart. Ablation studies further verified the validity of the rectified flow technique in VoiceFlow.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 31 of its 36 references that are in the map. First posted on arXiv 10 Sep 2023.

Builds on

31 of 36 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023LightGrad: Lightweight Diffusion Probabilistic Model for Text-to-Speech31 Aug
  2. 2023Boosting Fast and High-Quality Speech Synthesis with Linear Diffusion9 Jun
  3. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model11 May
  4. 2023DiffVoice: Text-to-Speech with Latent Diffusion23 Apr
  5. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech13 Jul
  6. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis21 Apr
  7. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis25 Mar
  8. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidance23 Nov
  9. 2021EdiTTS: Score-based Editing for Controllable Text-to-Speech6 Oct
  10. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme28 Sep
  11. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May

Other references · 20

  1. 2020Denoising Diffusion Implicit Modelscited by 9,781
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  5. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  6. 2023Consistency Modelscited by 1,822
  7. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  8. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
  9. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  10. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  11. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  12. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
8 more
  1. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  2. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  3. 2019MOSNet: Deep Learning based Objective Assessment for Voice Conversioncited by 127
  4. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 92
  5. 2020Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesiscited by 54
  6. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  7. 2023UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocodingcited by 37
  8. 2022VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Featurecited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 13; 2025: 27; 2026: 8.

49 papers.

Sort:

By area:

  1. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  2. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  3. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  4. 2024Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matchingcited by 53
  5. 2024AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policiescited by 50
  6. 2023Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesiscited by 30
  7. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matchingcited by 30
  8. 2024DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Datasetcited by 20
  9. 2024FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generationcited by 19
  10. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  11. 2025CoVoMix2: Advancing Zero-Shot Dialogue Generation with Fully Non-Autoregressive Flow Matchingcited by 11
  12. 2025FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generationcited by 11
37 more
  1. 2025Prosody-Enhanced Acoustic Pre-training and Acoustic-Disentangled Prosody Adapting for Movie Dubbingcited by 10
  2. 2025TARO: Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning for Synchronized Video-to-Audio Synthesiscited by 10
  3. 2025On the minimax optimality of Flow Matching through the connection to kernel density estimationcited by 10
  4. 2024E1 TTS: Simple and Fast Non-Autoregressive TTScited by 7
  5. 2025Step-Audio-EditX Technical Reportcited by 7
  6. 2024Small-E: Small Language Model with Linear Attention for Efficient Speech Synthesiscited by 6
  7. 2025FlowDubber: Movie Dubbing with LLM-based Semantic-aware Learning and Flow Matching based Voice Enhancingcited by 6
  8. 2025Distribution estimation via Flow Matching with Lipschitz guaranteescited by 6
  9. 2025Flow Straight and Fast in Hilbert Space: Functional Rectified Flowcited by 6
  10. 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalitiescited by 6
  11. 2024Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesiscited by 5
  12. 2024DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesiscited by 5
  13. 2025DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generationcited by 5
  14. 2025Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Samplingcited by 5
  15. 2024SF-Speech: Straightened Flow for Zero-Shot Voice Clonecited by 4
  16. 2025MusFlow: Multimodal Music Generation via Conditional Flow Matchingcited by 4
  17. 2025Weighted Conditional Flow Matchingcited by 4
  18. 2024LiteFocus: Accelerated Diffusion Inference for Long Audio Synthesiscited by 3
  19. 2025SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flowcited by 3
  20. 2025Diffusion Bridge or Flow Matching? A Unifying Framework and Comparative Analysiscited by 3
  21. 2025SyMuPe: Affective and Controllable Symbolic Music Performancecited by 3
  22. 2025Deep Learning-based Intrusion Detection Systems: A Surveycited by 2
  23. 2025Shallow Flow Matching for Coarse-to-Fine Text-to-Speech Synthesiscited by 2
  24. 2026DiFlowDubber: Discrete Flow Matching for Automated Video Dubbing via Cross-Modal Alignment and Synchronizationcited by 2
  25. 2026Few-shot Acoustic Synthesis with Multimodal Flow Matchingcited by 2
  26. 2025RapFlow-TTS: Rapid and High-Fidelity Text-to-Speech with Improved Consistency Flow Matchingcited by 1
  27. 2025Curved Worlds, Clear Boundaries: Generalizing Speech Deepfake Detection using Hyperbolic and Spherical Geometry Spacescited by 1
  28. 2026CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbingcited by 1
  29. 2026WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modelingcited by 1
  30. 2026Transcript-Free Flow-Matching Text-to-Speech via Speech Feature Conditioningcited by 1
  31. 2025Speaking images. A novel framework for the automated self-description of artworkscited by 0
  32. 2025Discrete-Time Diffusion-Like Models for Speech Synthesiscited by 0
  33. 2025Speak, Edit, Repeat: High-Fidelity Voice Editing and Zero-Shot TTS with Cross-Attentive Mambacited by 0
  34. 2025CogSR: Semantic-Aware Speech Super-Resolution via Chain-of-Thought Guided Flow Matchingcited by 0
  35. 2026FlowW2N: Whispered-to-Normal Speech Conversion via Flow-Matchingcited by 0
  36. 2026Fr'echet Distance Loss on Speech Representations for Text-to-Speech Synthesiscited by 0
  37. 2026SEmoEdit: Probing and Harnessing the Editability of Pre-trained Speech Flowscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speech
  2. 2024LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation
  3. 2025ReFlow-VC: Zero-shot Voice Conversion Based on Rectified Flow and Speaker Feature Optimization
  4. 2025AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion
  5. 2026MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flows
  6. 2025DiFlow-TTS: Compact and Low-Latency Zero-Shot Text-to-Speech with Discrete Flow Matching
  7. 2026DSFlow: Dual Supervision and Step-Aware Architecture for One-Step Flow Matching Speech Synthesis
  8. 2025OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching