mlsanityData to 14 Oct 2026

Shallow Flow Matching for Coarse-to-Fine Text-to-Speech Synthesis

arXiv 2505.12226 ↗First posted 18 May 2025eess.AS cs.AI cs.SDcited by 2

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We propose Shallow Flow Matching (SFM), a novel mechanism that enhances flow matching (FM)-based text-to-speech (TTS) models within a coarse-to-fine generation paradigm. Unlike conventional FM modules, which use the coarse representations from the weak generator as conditions, SFM constructs intermediate states along the FM paths from these representations. During training, we introduce an orthogonal projection method to adaptively determine the temporal position of these states, and apply a principled construction strategy based on a single-segment piecewise flow. The SFM inference starts from the intermediate state rather than pure noise, thereby focusing computation on the latter stages of the FM paths. We integrate SFM into multiple TTS models with a lightweight SFM head. Experiments demonstrate that SFM yields consistent gains in speech naturalness across both objective and subjective evaluations, and significantly accelerates inference when using adaptive-step ODE solvers. Demo and codes are available at https://ydqmkkx.github.io/SFMDemo/.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 43 of its 50 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

43 of 50 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization3 Apr
  2. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models13 Dec
  3. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  4. 2023E3 TTS: Easy End-to-End Diffusion-based Text to Speech2 Nov
  5. 2023ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speech29 Sep
  6. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching10 Sep
  7. 2023Matcha-TTS: A fast TTS architecture with conditional flow matching6 Sep
  8. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May

Other references · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  7. 2018Neural Ordinary Differential Equationscited by 3,779
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  9. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  10. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  11. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  12. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
23 more
  1. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  2. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  3. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  4. 2024One Step Diffusion via Shortcut Modelscited by 391
  5. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  6. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  7. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  8. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  9. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  10. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  11. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  12. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  13. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  14. 2024PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Acceleratorcited by 97
  15. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 92
  16. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
  17. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 75
  18. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 70
  19. 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65
  20. 2024Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesiscited by 48
  21. 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 48
  22. 2025Distillation and Pruning for Scalable Self-Supervised Representation-Based Speech Quality Assessmentcited by 13
  23. 2025LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLMcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025FloodDiffusion: Tailored Diffusion Forcing for Streaming Motion Generationcited by 16
  2. 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speechcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Flow-Matching-based TTS without Classifier-Free Guidance
  2. 2025RapFlow-TTS: Rapid and High-Fidelity Text-to-Speech with Improved Consistency Flow Matching
  3. 2026Fr'echet Distance Loss on Speech Representations for Text-to-Speech Synthesis
  4. 2026DSFlow: Dual Supervision and Step-Aware Architecture for One-Step Flow Matching Speech Synthesis
  5. 2025Accelerating Flow-Matching-Based Text-to-Speech via Empirically Pruned Step Sampling
  6. 2026BareWave: Waveform-Native Flow-Matching Text-to-Speech
  7. 2026ReaFlow-TTS: Realization-Conditioned Flow Matching for High-Quality and Controllable Speech Synthesis
  8. 2026Distill Locally, Schedule Globally: Flow Maps for Few-Step Text-to-Speech