mlsanityData to 14 Oct 2026

ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching

arXiv 2506.13053 ↗First posted 16 Jun 2025eess.AS cs.SDcited by 30

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Existing large-scale zero-shot text-to-speech (TTS) models deliver high speech quality but suffer from slow inference speeds due to massive parameters. To address this issue, this paper introduces ZipVoice, a high-quality flow-matching-based zero-shot TTS model with a compact model size and fast inference speed. Key designs include: 1) a Zipformer-based vector field estimator to maintain adequate modeling capabilities under constrained size; 2) Average upsampling-based initial speech-text alignment and Zipformer-based text encoder to improve speech intelligibility; 3) A flow distillation method to reduce sampling steps and eliminate the inference overhead associated with classifier-free guidance. Experiments on 100k hours multilingual datasets show that ZipVoice matches state-of-the-art models in speech quality, while being 3 times smaller and up to 30 times faster than a DiT-based flow-matching baseline. Codes, model checkpoints and demo samples are publicly available at https://github.com/k2-fsa/ZipVoice.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 51 of its 57 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

51 of 57 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flow10 Apr
  2. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models13 Dec
  3. 2024SF-Speech: Straightened Flow for Zero-Shot Voice Clone16 Oct
  4. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  5. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors17 Jun
  6. 2024SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Models4 Jun
  7. 2024FlashSpeech: Efficient Zero-Shot Speech Synthesis23 Apr
  8. 2023ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speech29 Sep
  9. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching10 Sep
  10. 2023Matcha-TTS: A fast TTS architecture with conditional flow matching6 Sep
  11. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model11 May
  12. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr
1 more
  1. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May

Other references · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Classifier-Free Diffusion Guidancecited by 5,847
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  8. 2023Consistency Modelscited by 1,822
  9. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  10. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  11. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  12. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
26 more
  1. 2022On Distillation of Guided Diffusion Modelscited by 725
  2. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  3. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  4. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  5. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  6. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  7. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  8. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  9. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  10. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  11. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  12. 2023FreeU: Free Lunch in Diffusion U-Netcited by 196
  13. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  14. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  15. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  16. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  17. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  18. 2023Libriheavy: a 50,000 hours ASR corpus with punctuation casing and contextcited by 85
  19. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 80
  20. 2024FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applicationscited by 68
  21. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformerscited by 54
  22. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 49
  23. 2020DiDiSpeech: A Large Scale Mandarin Speech Corpuscited by 33
  24. 2022Revisiting Over-Smoothness in Text to Speechcited by 32
  25. 2021LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Searchcited by 28
  26. 2023Amphion: An Open-Source Audio, Music and Speech Generation Toolkitcited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 23.

30 papers.

Sort:

By area:

  1. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Modelscited by 25
  2. 2025ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matchingcited by 7
  3. 2026MM-Sonate: Multimodal Controllable Audio-Video Generation with Zero-Shot Voice Cloningcited by 7
  4. 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesiscited by 6
  5. 2026IndexTTS 2.5 Technical Reportcited by 6
  6. 2026Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generationcited by 6
  7. 2025VoXtream: Full-Stream Text-to-Speech with Extremely Low Latencycited by 5
  8. 2025InstructAudio: Unified speech and music generation with natural language instructioncited by 5
  9. 2025Flow2GAN: Hybrid Flow Matching and GAN with Multi-Resolution Network for Few-step High-Fidelity Audio Generationcited by 4
  10. 2026MMControl: Unified Multi-Modal Control for Joint Audio-Video Generationcited by 4
  11. 2026Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesiscited by 3
  12. 2026VoXtream2: Full-stream TTS with dynamic speaking rate controlcited by 3
18 more
  1. 2026LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Spacecited by 3
  2. 2026Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTScited by 3
  3. 2026UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructionscited by 2
  4. 2025DiTReducio: A Training-Free Acceleration for DiT-Based TTS via Progressive Calibrationcited by 1
  5. 2026WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modelingcited by 1
  6. 2025SyncVoice: Simple and Effective Automatic Video Dubbing with Vision-Augmented TTScited by 0
  7. 2026What Counts as Real? Speech Restoration and Voice Quality Conversion Pose New Challenges to Deepfake Detectioncited by 0
  8. 2026Conditioned in Training, Dispensable at Inference: Typological Conditioning for Multilingual Speech Translationcited by 0
  9. 2026Towards Unified Song Generation and Singing Voice Conversion with Accompaniment Co-Generationcited by 0
  10. 2026End-to-End Training for Discrete Token LLM based TTS Systemcited by 0
  11. 2026Reference-Driven Multi-Speaker Audio Scene Generation from In-the-Wild Priorscited by 0
  12. 2026ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Modelscited by 0
  13. 2026FreyaTTS: A Compact Tokenizer-Free Flow-Matching Transformer for Turkish-First Speech Synthesiscited by 0
  14. 2026Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUscited by 0
  15. 2026Zero-Shot Face-to-Speech Synthesis via Latent Space Adaptation of a Style-Diffusion TTS Modelcited by 0
  16. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latentscited by 0
  17. 2026Not Quite My Tempo: Voice Activity-aware Speech Synthesis for Lip-Synchronous Dubbingcited by 0
  18. 2026ReaFlow-TTS: Realization-Conditioned Flow Matching for High-Quality and Controllable Speech Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DiFlow-TTS: Compact and Low-Latency Zero-Shot Text-to-Speech with Discrete Flow Matching
  2. 2025Flamed-TTS: Flow Matching Attention-Free Models for Efficient Generating and Dynamic Pacing Zero-shot Text-to-Speech
  3. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis
  4. 2026ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference
  5. 2025OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching
  6. 2025Towards Lightweight and Stable Zero-shot TTS with Self-distilled Representation Disentanglement
  7. 2025Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling
  8. 2024An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS