mlsanityData to 7 Oct 2026

RobustSpeechFlow: Learning Robust Text-to-Speech Trajectories via Augmentation-based Contrastive Flow Matching

arXiv 2605.22083 ↗First posted 21 May 2026cs.SD cs.LG eess.AScited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

While flow-matching text-to-speech (TTS) achieves strong zero-shot speaker similarity and naturalness, it remains susceptible to content fidelity issues, particularly skip and repeat errors from imperfect alignment. We propose RobustSpeechFlow, a training strategy that improves alignment robustness by extending contrastive flow matching with length-preserving repeat and skip latent augmentations. Requiring no external aligners or preference data, our method directly penalizes realistic failure modes and readily integrates into existing pipelines. On Seed-TTS-eval, it reduces the word error rate (WER) from 1.44 to 1.38 using only 0.06B parameters. On our ZERO500 benchmark, it delivers consistent intelligibility improvements across diverse speaker and prosody conditions; at NFE=24, it reduces English character error rate (CER) from 0.48% to 0.35% and Korean CER from 0.81% to 0.57%. Audio samples: https://robustspeechflow.github.io/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 24 of its 28 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

24 of 28 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training23 May
  2. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis26 Feb
  3. 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation6 Feb
  4. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  5. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors17 Jun

Other references · 19

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  6. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  7. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
  8. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 200
  9. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 109
  10. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 85
  11. 2017Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attentioncited by 57
  12. 2024Multimodal Latent Language Modeling with Next-Token Diffusioncited by 47
7 more
  1. 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 44
  2. 2025VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloningcited by 39
  3. 2025Contrastive Flow Matchingcited by 29
  4. 2025Advancing Zero-shot Text-to-Speech Intelligibility across Diverse Domains via Preference Alignmentcited by 18
  5. 2024DualSpeech: Enhancing Speaker-Fidelity and Text-Intelligibility Through Dual Classifier-Free Guidancecited by 11
  6. 2025Length-Aware Rotary Position Embedding for Text-Speech Alignmentcited by 2
  7. 2025SupertonicTTS: Towards Highly Efficient and Streamlined Text-to-Speech Systemcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference
  2. 2025Time-Layer Adaptive Alignment for Speaker Similarity in Flow-Matching Based Zero-Shot TTS ↗
  3. 2026PFluxTTS: Hybrid Flow-Matching TTS with Robust Cross-Lingual Voice Cloning and Inference-Time Model Fusion
  4. 2025DiFlow-TTS: Compact and Low-Latency Zero-Shot Text-to-Speech with Discrete Flow Matching
  5. 2025OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching
  6. 2026Enhancing Flow Matching with A Unified Guidance Framework for Efficient and Robust Speech Synthesis
  7. 2024Zero-Shot Voice Conversion via Content-Aware Timbre Ensemble and Conditional Flow Matching
  8. 2024An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS