mlsanityData to 7 Oct 2026

Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling

arXiv 2505.19931 ↗First posted 26 May 2025eess.AS cs.SDcited by 5

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Flow-matching-based text-to-speech (TTS) models, such as Voicebox, E2 TTS, and F5-TTS, have attracted significant attention in recent years. These models require multiple sampling steps to reconstruct speech from noise, making inference speed a key challenge. Reducing the number of sampling steps can greatly improve inference efficiency. To this end, we introduce Fast F5-TTS, a training-free approach to accelerate the inference of flow-matching-based TTS models. By inspecting the sampling trajectory of F5-TTS, we identify redundant steps and propose Empirically Pruned Step Sampling (EPSS), a non-uniform time-step sampling strategy that effectively reduces the number of sampling steps. Our approach achieves a 7-step generation with an inference RTF of 0.030 on an NVIDIA RTX 3090 GPU, making it 4 times faster than the original F5-TTS while maintaining comparable performance. Furthermore, EPSS performs well on E2 TTS models, demonstrating its strong generalization ability.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 30 of its 31 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

30 of 31 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models13 Dec
  2. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  3. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesis8 Jun
  4. 2024FlashSpeech: Efficient Zero-Shot Speech Synthesis23 Apr
  5. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models5 Mar
  6. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching10 Sep
  7. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr

Other references · 23

  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  7. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  8. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  9. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  10. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  11. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  12. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
11 more
  1. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  2. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
  3. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
  4. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  5. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 150
  6. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 149
  7. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
  8. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 118
  9. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
  10. 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65
  11. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesiscited by 6
  2. 2026WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modelingcited by 1
  3. 2026Audio Deepfake Detection in the Age of Advanced Text-to-Speech modelscited by 0
  4. 2026Beyond Two-stage Diffusion TTS: Joint Structure and Content Refinement via Jump Diffusioncited by 0
  5. 2026Distill Locally, Schedule Globally: Flow Maps for Few-Step Text-to-Speechcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Flow-Matching-based TTS without Classifier-Free Guidance
  2. 2025RapFlow-TTS: Rapid and High-Fidelity Text-to-Speech with Improved Consistency Flow Matching
  3. 2026Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUs
  4. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
  5. 2025OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching
  6. 2026Fr'echet Distance Loss on Speech Representations for Text-to-Speech Synthesis
  7. 2026FlashTTS: Fast Streaming TTS with MTP Acceleration and X-pred Mean Flow Distillation
  8. 2026DSFlow: Dual Supervision and Step-Aware Architecture for One-Step Flow Matching Speech Synthesis