mlsanityData to 7 Oct 2026

OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models

arXiv 2604.00688 ↗First posted 1 Apr 2026cs.CL eess.AScited by 22

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We present OmniVoice, a massively multilingual zero-shot text-to-speech (TTS) model that scales to over 600 languages. At its core is a novel diffusion language model-style discrete non-autoregressive (NAR) architecture. Unlike conventional discrete NAR models that suffer from performance bottlenecks in complex two-stage (text-to-semantic-to-acoustic) pipelines, OmniVoice directly maps text to multi-codebook acoustic tokens. This simplified approach is facilitated by two key technical innovations: (1) a full-codebook random masking strategy for efficient training, and (2) initialization from a pre-trained LLM to ensure superior intelligibility. By leveraging a 581k-hour multilingual dataset curated entirely from open-source data, OmniVoice achieves the broadest language coverage to date and delivers state-of-the-art performance across Chinese, English, and diverse multilingual benchmarks. Our code and pre-trained models are publicly available at https://github.com/k2-fsa/OmniVoice.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 79 of its 92 references that are in the map. First posted on arXiv 1 Apr 2026.

Builds on

79 of 92 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching12 Jul
  2. 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching16 Jun
  3. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training23 May
  4. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis26 Feb
  5. 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation6 Feb
  6. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models13 Dec
  7. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct

Other references · 72

  1. 2017Attention Is All You Needcited by 59,932
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  7. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  8. 2025Large Language Diffusion Modelscited by 872
  9. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  10. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  11. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  12. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
38 more
  1. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  2. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
  3. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  4. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  5. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  6. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
  7. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  8. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 174
  9. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  10. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 149
  11. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
  12. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 118
  13. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  14. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 109
  15. 2026Qwen3-TTS Technical Reportcited by 106
  16. 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 85
  17. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 69
  18. 2024FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applicationscited by 68
  19. 2017Speech Recognition Challenge in the Wild: Arabic MGB-3cited by 60
  20. 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 46
  21. 2024Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesiscited by 45
  22. 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 44
  23. 2025VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloningcited by 39
  24. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  25. 2025Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languagescited by 32
  26. 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
  27. 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 27
  28. 2021The Accented English Speech Recognition Challenge 2020: Open Datasets, Tracks, Baselines, Results and Methodscited by 24
  29. 2016The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognitioncited by 21
  30. 2025Granary: Speech Recognition and Translation Dataset in 25 European Languagescited by 21
  31. 2025Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generationcited by 19
  32. 2025Scaling Rich Style-Prompted Text-to-Speech Datasetscited by 17
  33. 2023CML-TTS A Multilingual Dataset for Speech Synthesis in Low-Resource Languagescited by 15
  34. 2024IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTScited by 15
  35. 2023ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpuscited by 14
  36. 2025NVSpeech: An Integrated and Scalable Pipeline for Human-Like Speech Modeling with Paralinguistic Vocalizationscited by 14
  37. 2024Mixat: A Data Set of Bilingual Emirati-English Speechcited by 12
  38. 2025A Scalable Pipeline for Enabling Non-Verbal Speech Generation and Understandingcited by 11

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 22.

22 papers.

Sort:

By area:

  1. 2026Toward Native Multimodal Modeling: A Roadmapcited by 6
  2. 2026dots.tts Technical Reportcited by 3
  3. 2026Praxy Voice: Voice-Prompt Recovery + BUPS for Commercial-Class Indic TTS from a Frozen Non-Indic Base at Zero Commercial-Training-Data Costcited by 1
  4. 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speechcited by 1
  5. 2026WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modelingcited by 1
  6. 2026How to Leverage Synthetic Speech for LLM-Based ASR Systems?cited by 1
  7. 2026AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editingcited by 1
  8. 2026End-to-End Training for Discrete Token LLM based TTS Systemcited by 0
  9. 2026LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answeringcited by 0
  10. 2026FreyaTTS: A Compact Tokenizer-Free Flow-Matching Transformer for Turkish-First Speech Synthesiscited by 0
  11. 2026Do Speech Tokens Leak Voiceprints? Speaker Inversion Attacks Against End-to-End Speech Language Modelscited by 0
  12. 2026ESCUCHA: A Spanish Speech Benchmark for Heterogeneous Acoustic Conditionscited by 0
10 more
  1. 2026MyMentorLLM: A psychotherapy GenAI environment with multimodal voice/text patients, trainees and experts for deliberate practicecited by 0
  2. 2026Analyzing Speech Condition Effects in Dysarthric ASR: A Layer-wise Probing Studycited by 0
  3. 2026Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encodercited by 0
  4. 2026FastThaiG2P: Lightning-fast Thai Grapheme-to-phoneme Conversion for Voice Agent Pipelinescited by 0
  5. 2026Building and Evaluating a Synthetic Bengali Speech Resource for Telecom Customer Carecited by 0
  6. 2026Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Modelscited by 0
  7. 2026Complex-Text Robustness Evaluation and Failure Diagnosis for Low-Resource Multilingual Text-to-Speechcited by 0
  8. 2026Dictionary-Constrained Grapheme-to-Phoneme for Unsegmented Languages from LLM-Annotated Datacited by 0
  9. 2026Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architecturescited by 0
  10. 2026Accent Analogy Guidance: More Speaker Similarity at Equal Accent in Cross-Lingual Voice Cloningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias
  2. 2026Raon-OpenTTS: Open Models and Data for Robust Text-to-Speech ↗
  3. 2026LLaDA-TTS: Unifying Speech Synthesis and Zero-Shot Editing via Masked Diffusion Modeling
  4. 2024Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation
  5. 2026Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion
  6. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  7. 2025OmniFusion: Simultaneous Multilingual Multimodal Translations via Modular Fusion
  8. 2025From Text to Talk: Audio-Language Model Needs Non-Autoregressive Joint Training