# OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models

arXiv: https://arxiv.org/abs/2604.00688 · HTML: https://mlsanity.com/paper/2604.00688/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 79 of its 92 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 30 of 79)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching](https://mlsanity.com/paper/2410.06885/) (2024)
- [Seed-TTS: A Family of High-Quality Versatile Speech Generation Models](https://mlsanity.com/paper/2406.02430/) (2024)
- [CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models](https://mlsanity.com/paper/2412.10117/) (2024)
- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model](https://mlsanity.com/paper/2406.04904/) (2024)
- [MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer](https://mlsanity.com/paper/2409.00750/) (2024)
- [E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS](https://mlsanity.com/paper/2406.18009/) (2024)
- [CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training](https://mlsanity.com/paper/2505.17589/) (2025)
- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs](https://mlsanity.com/paper/2407.04051/) (2024)
- [Qwen3-TTS Technical Report](https://mlsanity.com/paper/2601.15621/) (2026)
- [SoundStorm: Efficient Parallel Audio Generation](https://mlsanity.com/paper/2305.09636/) (2023)
- [Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens](https://mlsanity.com/paper/2503.01710/) (2025)

## Built on by (most cited first, 25 of 25)

- [Toward Native Multimodal Modeling: A Roadmap](https://mlsanity.com/paper/2605.25343/) (2026)
- [dots.tts Technical Report](https://mlsanity.com/paper/2606.07080/) (2026)
- [Praxy Voice: Voice-Prompt Recovery + BUPS for Commercial-Class Indic TTS from a Frozen Non-Indic Base at Zero Commercial-Training-Data Cost](https://mlsanity.com/paper/2604.25441/) (2026)
- [Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speech](https://mlsanity.com/paper/2605.09386/) (2026)
- [WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling](https://mlsanity.com/paper/2606.03455/) (2026)
- [How to Leverage Synthetic Speech for LLM-Based ASR Systems?](https://mlsanity.com/paper/2606.29031/) (2026)
- [AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing](https://mlsanity.com/paper/2609.08936/) (2026)
- [End-to-End Training for Discrete Token LLM based TTS System](https://mlsanity.com/paper/2606.09234/) (2026)
- [LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering](https://mlsanity.com/paper/2607.02763/) (2026)
- [FreyaTTS: A Compact Tokenizer-Free Flow-Matching Transformer for Turkish-First Speech Synthesis](https://mlsanity.com/paper/2607.09530/) (2026)
- [Do Speech Tokens Leak Voiceprints? Speaker Inversion Attacks Against End-to-End Speech Language Models](https://mlsanity.com/paper/2607.16870/) (2026)
- [ESCUCHA: A Spanish Speech Benchmark for Heterogeneous Acoustic Conditions](https://mlsanity.com/paper/2607.17812/) (2026)
- [MyMentorLLM: A psychotherapy GenAI environment with multimodal voice/text patients, trainees and experts for deliberate practice](https://mlsanity.com/paper/2607.25667/) (2026)
- [Analyzing Speech Condition Effects in Dysarthric ASR: A Layer-wise Probing Study](https://mlsanity.com/paper/2608.01865/) (2026)
- [Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder](https://mlsanity.com/paper/2608.11650/) (2026)
- [FastThaiG2P: Lightning-fast Thai Grapheme-to-phoneme Conversion for Voice Agent Pipelines](https://mlsanity.com/paper/2608.12814/) (2026)
- [Building and Evaluating a Synthetic Bengali Speech Resource for Telecom Customer Care](https://mlsanity.com/paper/2608.20346/) (2026)
- [Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models](https://mlsanity.com/paper/2609.01723/) (2026)
- [Complex-Text Robustness Evaluation and Failure Diagnosis for Low-Resource Multilingual Text-to-Speech](https://mlsanity.com/paper/2609.11545/) (2026)
- [Dictionary-Constrained Grapheme-to-Phoneme for Unsegmented Languages from LLM-Annotated Data](https://mlsanity.com/paper/2609.19805/) (2026)
- [Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures](https://mlsanity.com/paper/2609.28974/) (2026)
- [Accent Analogy Guidance: More Speaker Similarity at Equal Accent in Cross-Lingual Voice Cloning](https://mlsanity.com/paper/2609.29123/) (2026)
- [A Comprehensive Objective Evaluation of Modern Text-to-Speech for Turkish Using Speech Quality Assessment Models](https://mlsanity.com/paper/2610.06057/) (2026)
- [BabelFake: A Multilingual Audio-Visual DeepFake Benchmark](https://mlsanity.com/paper/2610.06339/) (2026)
- [Fish Audio Drama 3 Technical Report](https://mlsanity.com/paper/2610.18010/) (2026)

## Abstract (arXiv)

We present OmniVoice, a massively multilingual zero-shot text-to-speech (TTS) model that scales to over 600 languages. At its core is a novel diffusion language model-style discrete non-autoregressive (NAR) architecture. Unlike conventional discrete NAR models that suffer from performance bottlenecks in complex two-stage (text-to-semantic-to-acoustic) pipelines, OmniVoice directly maps text to multi-codebook acoustic tokens. This simplified approach is facilitated by two key technical innovations: (1) a full-codebook random masking strategy for efficient training, and (2) initialization from a pre-trained LLM to ensure superior intelligibility. By leveraging a 581k-hour multilingual dataset curated entirely from open-source data, OmniVoice achieves the broadest language coverage to date and delivers state-of-the-art performance across Chinese, English, and diverse multilingual benchmarks. Our code and pre-trained models are publicly available at https://github.com/k2-fsa/OmniVoice.