mlsanityData to 14 Oct 2026

Transcript-Free Flow-Matching Text-to-Speech via Speech Feature Conditioning

arXiv 2606.20266 ↗First posted 18 Jun 2026eess.AScited by 1

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recent flow-matching text-to-speech (TTS) models, such as F5-TTS, rely on a reference transcript at inference time, obtained from an external ASR system. This dependency makes zero-shot TTS brittle for accented or dysarthric speakers, precisely the scenarios where it is most needed. Moreover, we find that text-based reference conditioning can propagate atypical acoustic patterns from atypical speech into synthesis, even when ground-truth transcripts are available. To address this, we propose RTFree-F5, which replaces the reference transcript with continuous self-supervised speech representations mapped into F5-TTS's text-conditioning space via a lightweight adapter, while reusing the pretrained checkpoint. On dysarthric speech, RTFree-F5 reduces WER from 24.6% to 10.4%, surpassing even the ground-truth reference transcript baselines, while improving naturalness and remaining competitive on standard benchmarks without requiring any reference transcript.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 30 of its 31 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

30 of 31 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025DiffDSR: Dysarthric Speech Reconstruction Using Latent Diffusion Model31 May
  2. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  3. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors17 Jun
  4. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching10 Sep
  5. 2023Matcha-TTS: A fast TTS architecture with conditional flow matching6 Sep
  6. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr

Other references · 24

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2016Layer Normalizationcited by 5,669
  7. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  8. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  9. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  10. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  11. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  12. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
12 more
  1. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  2. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  3. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  4. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  5. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  6. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  7. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 122
  8. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 49
  9. 2025The Interspeech 2025 Speech Accessibility Project Challengecited by 9
  10. 2024UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalizationcited by 8
  11. 2025EZ-VC: Easy Zero-shot Any-to-Any Voice Conversioncited by 2
  12. 2025SpeechAccentLLM: A Unified Framework for Foreign Accent Conversion and Text to Speechcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encodercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference
  2. 2025Improved Intelligibility of Dysarthric Speech using Conditional Flow Matching
  3. 2025OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching
  4. 2026Mask, Sample, Revise: A Revisable CTMC Inference Stack for Guided Discrete Flow Matching Text-to-Speech
  5. 2026Fr'echet Distance Loss on Speech Representations for Text-to-Speech Synthesis
  6. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  7. 2025Towards Flow-Matching-based TTS without Classifier-Free Guidance
  8. 2026PFluxTTS: Hybrid Flow-Matching TTS with Robust Cross-Lingual Voice Cloning and Inference-Time Model Fusion