mlsanityData to 7 Oct 2026

Qwen3-TTS Technical Report

arXiv 2601.15621 ↗First posted 22 Jan 2026cs.SD cs.CL eess.AScited by 106

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this report, we present the Qwen3-TTS series, a family of advanced multilingual, controllable, robust, and streaming text-to-speech models. Qwen3-TTS supports state-of-the-art 3-second voice cloning and description-based control, allowing both the creation of entirely novel voices and fine-grained manipulation over the output speech. Trained on over 5 million hours of speech data spanning 10 languages, Qwen3-TTS adopts a dual-track LM architecture for real-time synthesis, coupled with two speech tokenizers: 1) Qwen-TTS-Tokenizer-25Hz is a single-codebook codec emphasizing semantic content, which offers seamlessly integration with Qwen-Audio and enables streaming waveform reconstruction via a block-wise DiT. 2) Qwen-TTS-Tokenizer-12Hz achieves extreme bitrate reduction and ultra-low-latency streaming, enabling immediate first-packet emission (97,ms) through its 12.5 Hz, 16-layer multi-codebook design and a lightweight causal ConvNet. Extensive experiments indicate state-of-the-art performance across diverse objective and subjective benchmark (e.g., TTS multilingual test set, InstructTTSEval, and our long speech test set). To facilitate community research and development, we release both tokenizers and models under the Apache 2.0 license.

Key facts. Cited by 106 papers in the map (data through 7 Oct 2026). Builds on 41 of its 43 references that are in the map. First posted on arXiv 22 Jan 2026.

Builds on

41 of 43 references.

References · 41

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  4. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  5. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  6. 2025Qwen2.5-Omni Technical Reportcited by 704
  7. 2022High Fidelity Neural Audio Compressioncited by 673
  8. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  9. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  10. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  11. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
  12. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
29 more
  1. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  2. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  3. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  4. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  5. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  6. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 149
  7. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 142
  8. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  9. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 109
  10. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 106
  11. 2023SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Modelscited by 85
  12. 2024Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Modelcited by 70
  13. 2025Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesiscited by 69
  14. 2024Natural language guidance of high-fidelity text-to-speech with synthetic annotationscited by 66
  15. 2025MiMo-Audio: Audio Language Models are Few-Shot Learnerscited by 58
  16. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
  17. 2024Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesiscited by 45
  18. 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 44
  19. 2025VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloningcited by 39
  20. 2025FireRedTTS-2: Towards Long Conversational Speech Generation for Podcast and Chatbotcited by 34
  21. 2023PromptStyle: Controllable Style Transfer for Text-to-Speech with Natural Language Descriptionscited by 33
  22. 2025VibeVoice Technical Reportcited by 28
  23. 2025XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecscited by 21
  24. 2024VoxInstruct: Expressive Human Instruction-to-Speech Generation with Unified Multilingual Codec Language Modellingcited by 20
  25. 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Controlcited by 17
  26. 2025InstructTTSEval: Benchmarking Complex Natural-Language Instruction Following in Text-to-Speech Systemscited by 14
  27. 2026VoiceSculptor: Your Voice, Designed By Youcited by 8
  28. 2024KALL-E:Autoregressive Speech Synthesis with Next-Distribution Predictioncited by 5
  29. 2025StreamFlow: Streaming Flow Matching with Block-wise Guided Attention Mask for Speech Token Decodingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 106.

106 papers.

Sort:

By area:

  1. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Modelscited by 22
  2. 2026MOSS-TTS Technical Reportcited by 10
  3. 2026Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoderscited by 7
  4. 2026Audio2Tool: Speak, Call, Act -- A Dataset for Benchmarking Speech Tool Usecited by 7
  5. 2026Raon-Speech Technical Reportcited by 4
  6. 2026SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesiscited by 3
  7. 2026LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Spacecited by 3
  8. 2026ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Modelscited by 3
  9. 2026dots.tts Technical Reportcited by 3
  10. 2026VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agentscited by 3
  11. 2026Multimodal Duplex Interaction Agentcited by 3
  12. 2026Ex-Omni: Enabling 3D Facial Animation Generation for Omni-modal Large Language Modelscited by 2
38 more
  1. 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Modelscited by 2
  2. 2026TriageSim: A Conversational Emergency Triage Simulation Framework from Structured Electronic Health Recordscited by 2
  3. 2026MOSS-VoiceGenerator: Create Realistic Voices with Natural Language Descriptionscited by 2
  4. 2026Dynin-Omni: Omnimodal Unified Large Diffusion Language Modelcited by 2
  5. 2026TTS-PRISM: A Perceptual Reasoning and Interpretable Speech Model for Fine-Grained Diagnosiscited by 2
  6. 2026Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modelingcited by 2
  7. 2026Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generationcited by 2
  8. 2026Learning When to Think While Listening in Large Audio-Language Modelscited by 2
  9. 2026Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPOcited by 2
  10. 2026Hear2Act: Benchmarking When Prosody Should Change What an Assistant Doescited by 2
  11. 2026Preference Optimization for Non-Verbal Vocalization Synthesiscited by 2
  12. 2026Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Inputcited by 1
  13. 2026AgentSearchBench: A Benchmark for AI Agent Search in the Wildcited by 1
  14. 2026The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluationcited by 1
  15. 2026VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singingcited by 1
  16. 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speechcited by 1
  17. 2026LaSR: Context-Aware Speech Recognition via Latent Reasoningcited by 1
  18. 2026Sympatheia: Emotionally Adaptive Voice Assistant with Continuous Affect Conditioningcited by 1
  19. 2026JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curationcited by 1
  20. 2026WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modelingcited by 1
  21. 2026MagpieTTS-LF: Inference-Time Long-Form Speech Generation Without Training on Long-Form datacited by 1
  22. 2026An Evaluation Framework for Text-to-Speech Voice Reconstructioncited by 1
  23. 2026Bagpiper-TTS: Natural Language Guided Universal Speech Synthesiscited by 1
  24. 2026SpeechEQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Modelscited by 1
  25. 2026HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speechcited by 1
  26. 2026How to Leverage Synthetic Speech for LLM-Based ASR Systems?cited by 1
  27. 2026DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speechcited by 1
  28. 2026WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTScited by 1
  29. 2026VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentationcited by 1
  30. 2026FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generationcited by 1
  31. 2026ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansioncited by 1
  32. 2026AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editingcited by 1
  33. 2026CLASH: Counterfactual Auditing of Lexical and Prosodic Reliance in Spoken Sarcasm Detectioncited by 1
  34. 2026OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agentscited by 1
  35. 2026Investigating Group Relative Policy Optimization for Diffusion Transformer based Text-to-Audio Generationcited by 0
  36. 2026Knowing What to Stress: A Discourse-Conditioned Text-to-Speech Benchmarkcited by 0
  37. 2026V.O.I.C.E (Voice, Ownership, Identity, Control, Expression): Risk Taxonomy of Synthetic Voice Generation From Empirical Datacited by 0
  38. 2026EdgeFM: Efficient Edge Inference for Vision-Language Modelscited by 0

50 most cited of 106 All 106 (JSON)