mlsanityData to 14 Oct 2026

CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model

arXiv 2305.06908 ↗First posted 11 May 2023cs.SD cs.AI cs.CLcited by 45

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Denoising diffusion probabilistic models (DDPMs) have shown promising performance for speech synthesis. However, a large number of iterative steps are required to achieve high sample quality, which restricts the inference speed. Maintaining sample quality while increasing sampling speed has become a challenging task. In this paper, we propose a "Co"nsistency "Mo"del-based "Speech" synthesis method, CoMoSpeech, which achieve speech synthesis through a single diffusion sampling step while achieving high audio quality. The consistency constraint is applied to distill a consistency model from a well-designed diffusion-based teacher model, which ultimately yields superior performances in the distilled CoMoSpeech. Our experiments show that by generating audio recordings by a single sampling step, the CoMoSpeech achieves an inference speed more than 150 times faster than real-time on a single NVIDIA A100 GPU, which is comparable to FastSpeech2, making diffusion-sampling based speech synthesis truly practical. Meanwhile, objective and subjective evaluations on text-to-speech and singing voice synthesis show that the proposed teacher models yield the best audio quality, and the one-step sampling based CoMoSpeech achieves the best inference speed with better or comparable audio quality to other conventional multi-step diffusion model baselines. Audio samples are available at https://comospeech.github.io/.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 31 of its 42 references that are in the map. First posted on arXiv 11 May 2023.

Builds on

31 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr
  2. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan
  3. 2022ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speech30 Dec
  4. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech13 Jul
  5. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs28 Jan
  6. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May

Other references · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  7. 2023Consistency Modelscited by 1,822
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  9. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  10. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  11. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  12. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
13 more
  1. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  2. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  3. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 445
  4. 2021A Survey on Neural Speech Synthesiscited by 184
  5. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  6. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  7. 2023Monocular Depth Estimation using Diffusion Modelscited by 87
  8. 2023Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistentcited by 58
  9. 2022Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesiscited by 53
  10. 2020HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesiscited by 39
  11. 2019DurIAN: Duration Informed Attention Network For Multimodal Synthesiscited by 33
  12. 2020XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis Systemcited by 33
  13. 2019Sequence-to-sequence Singing Synthesis Using the Feed-forward Transformercited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 21; 2025: 13; 2026: 4.

45 papers.

Sort:

By area:

  1. 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 145
  2. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matchingcited by 49
  3. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  4. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  5. 2024Music2Latent: Consistency Autoencoders for Latent Audio Compressioncited by 33
  6. 2023Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesiscited by 30
  7. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matchingcited by 30
  8. 2024FlashSpeech: Efficient Zero-Shot Speech Synthesiscited by 24
  9. 2024LAFMA: A Latent Flow Matching Model for Text-to-Audio Generationcited by 24
  10. 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillationcited by 22
  11. 2023ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speechcited by 22
  12. 2024AudioLCM: Text-to-Audio Generation with Latent Consistency Modelscited by 21
33 more
  1. 2024CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Modelscited by 13
  2. 2024CoSIGN: Few-Step Guidance of ConSIstency Model to Solve General INverse Problemscited by 13
  3. 2024CoMoSVC: Consistency Model-based Singing Voice Conversioncited by 11
  4. 2024Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decodingcited by 10
  5. 2024E1 TTS: Simple and Fast Non-Autoregressive TTScited by 7
  6. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 7
  7. 2025CoDiCodec: Unifying Continuous and Discrete Compressed Representations of Audiocited by 7
  8. 2024Efficient Text-driven Motion Generation via Latent Consistency Trainingcited by 6
  9. 2024FastSAG: Towards Fast Non-Autoregressive Singing Accompaniment Generationcited by 5
  10. 2024DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variabilitycited by 5
  11. 2024DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesiscited by 5
  12. 2024DiffStyleTTS: Diffusion-based Hierarchical Prosody Modeling for Text-to-Speech with Diverse and Controllable Stylescited by 5
  13. 2025FlashSR: One-step Versatile Audio Super-resolution via Diffusion Distillationcited by 5
  14. 2024Accelerating High-Fidelity Waveform Generation via Adversarial Flow Matching Optimizationcited by 4
  15. 2025Music2Latent2: Audio Compression with Summary Embeddings and Autoregressive Decodingcited by 4
  16. 2024LiteFocus: Accelerated Diffusion Inference for Long Audio Synthesiscited by 3
  17. 2025SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flowcited by 3
  18. 2025Inference-time Scaling for Diffusion-based Audio Super-resolutioncited by 3
  19. 2025CoMelSinger: Discrete Token-Based Zero-Shot Singing Synthesis With Structured Melody Control and Guidancecited by 3
  20. 2025Towards Flow-Matching-based TTS without Classifier-Free Guidancecited by 2
  21. 2025APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speechcited by 2
  22. 2024DPI-TTS: Directional Patch Interaction for Fast-Converging and Style Temporal Modeling in Text-to-Speechcited by 1
  23. 2024EmoSpeech: A Corpus of Emotionally Rich and Contextually Detailed Speech Annotationscited by 1
  24. 2025Elucidating the Preconditioning in Consistency Distillationcited by 1
  25. 2025RapFlow-TTS: Rapid and High-Fidelity Text-to-Speech with Improved Consistency Flow Matchingcited by 1
  26. 2023Speech Synthesis By Unrolling Diffusion Process using Neural Network Layerscited by 0
  27. 2024ConSinger: Efficient High-Fidelity Singing Voice Generation with Minimal Stepscited by 0
  28. 2025Flamed-TTS: Flow Matching Attention-Free Models for Efficient Generating and Dynamic Pacing Zero-shot Text-to-Speechcited by 0
  29. 2025ECTSpeech: Enhancing Efficient Speech Synthesis via Easy Consistency Tuningcited by 0
  30. 2026Mitigating Latent Mismatch in cVAE-Based Singing Voice Synthesis via Flow Matchingcited by 0
  31. 2026DARS: Dysarthria-Aware Rhythm-Style Synthesis for ASR Enhancementcited by 0
  32. 2026Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesiscited by 0
  33. 2026DriftTTS: Few-Step Text-to-Speech Without Distillation via Distribution-Matching Driftcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme
  2. 2024RDSinger: Reference-based Diffusion Network for Singing Voice Synthesis
  3. 2025SmoothSinger: A Conditional Diffusion Model for Singing Voice Synthesis with Multi-Resolution Architecture
  4. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis
  5. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis
  6. 2024Speaking in Wavelet Domain: A Simple and Efficient Approach to Speed up Speech Diffusion Model
  7. 2023Boosting Fast and High-Quality Speech Synthesis with Linear Diffusion
  8. 2023HiddenSinger: High-Quality Singing Voice Synthesis via Neural Audio Codec and Latent Diffusion Models