mlsanityData to 14 Oct 2026

IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech

arXiv 2506.21619 ↗First posted 23 Jun 2025cs.CL cs.AI cs.SDcited by 90

Not on the map. Why?

Abstract (arXiv)

Existing autoregressive large-scale text-to-speech (TTS) models have advantages in speech naturalness, but their token-by-token generation mechanism makes it difficult to precisely control the duration of synthesized speech. This becomes a significant limitation in applications requiring strict audio-visual synchronization, such as video dubbing. This paper introduces IndexTTS2, which proposes a novel, general, and autoregressive model-friendly method for speech duration control. The method supports two generation modes: one explicitly specifies the number of generated tokens to precisely control speech duration; the other freely generates speech in an autoregressive manner without specifying the number of tokens, while faithfully reproducing the prosodic features of the input prompt. Furthermore, IndexTTS2 achieves disentanglement between emotional expression and speaker identity, enabling independent control over timbre and emotion. In the zero-shot setting, the model can accurately reconstruct the target timbre (from the timbre prompt) while perfectly reproducing the specified emotional tone (from the style prompt). To enhance speech clarity in highly emotional expressions, we incorporate GPT latent representations and design a novel three-stage training paradigm to improve the stability of the generated speech. Additionally, to lower the barrier for emotional control, we designed a soft instruction mechanism based on text descriptions by fine-tuning Qwen3, effectively guiding the generation of speech with the desired emotional orientation. Finally, experimental results on multiple datasets show that IndexTTS2 outperforms state-of-the-art zero-shot TTS models in terms of word error rate, speaker similarity, and emotional fidelity. Audio samples are available at: https://index-tts.github.io/index-tts2.github.io/

Key facts. Cited by 90 papers in the map (data through 14 Oct 2026). Builds on 54 of its 67 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

54 of 67 references.

References · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  9. 2018Neural Ordinary Differential Equationscited by 3,779
  10. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  11. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  12. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
38 more
  1. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  2. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440
  3. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  4. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  5. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  6. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  7. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  8. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  9. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  10. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  11. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 251
  12. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
  13. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 205
  14. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  15. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
  16. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  17. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  18. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  19. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 121
  20. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  21. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  22. 2023FunASR: A Fundamental End-to-End Speech Recognition Toolkitcited by 95
  23. 2024FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applicationscited by 68
  24. 2024Fish-Speech: Leveraging Large Language Models for Advanced Multilingual Text-to-Speech Synthesiscited by 48
  25. 2018Beyond One-hot Encoding: lower dimensional target embeddingcited by 43
  26. 2024Zero-shot Voice Conversion with Diffusion Transformerscited by 43
  27. 2023Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesiscited by 41
  28. 2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
  29. 2025Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglementcited by 35
  30. 2020DiDiSpeech: A Large Scale Mandarin Speech Corpuscited by 33
  31. 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 28
  32. 2024SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Modelscited by 23
  33. 2024VoxInstruct: Expressive Human Instruction-to-Speech Generation with Unified Multilingual Codec Language Modellingcited by 21
  34. 2024EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vectorcited by 19
  35. 2024EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbingcited by 17
  36. 2019A Survey on LoRa Networking: Research Problems, Current Solutions and Open Issuescited by 14
  37. 2024DubWise: Video-Guided Speech Duration Control in Multimodal LLM-based Text-to-Speech for Dubbingcited by 10
  38. 2024Takin: A Cohort of Superior Quality Zero-shot Speech Generation Modelscited by 7

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 85.

90 papers.

Sort:

By area:

  1. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Modelscited by 25
  2. 2026CoCoEmo: Composable and Controllable Human-Like Emotional TTS via Activation Steeringcited by 9
  3. 2025Evaluating Bias in Spoken Dialogue LLMs for Real-World Decisions and Recommendationscited by 7
  4. 2026MM-Sonate: Multimodal Controllable Audio-Video Generation with Zero-Shot Voice Cloningcited by 7
  5. 2026JUST-DUB-IT: Video Dubbing via Joint Audio-Visual Diffusioncited by 7
  6. 2026IndexTTS 2.5 Technical Reportcited by 6
  7. 2026WavBench: Benchmarking Reasoning, Colloquialism, and Paralinguistics for End-to-End Spoken Dialogue Modelscited by 6
  8. 2025Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generationcited by 5
  9. 2026FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speechcited by 5
  10. 2026X-Voice: Enabling Everyone to Speak 30 Languages via Zero-Shot Cross-Lingual Voice Cloningcited by 4
  11. 2025Soul: Breathe Life into Digital Human for High-fidelity Long-term Multimodal Animationcited by 3
  12. 2025Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesiscited by 3
38 more
  1. 2026FunCineForge: A Unified Dataset Toolkit and Model for Zero-Shot Movie Dubbing in Diverse Cinematic Scenescited by 3
  2. 2026SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesiscited by 3
  3. 2026VoXtream2: Full-stream TTS with dynamic speaking rate controlcited by 3
  4. 2026MOSS-VoiceGenerator: Create Realistic Voices with Natural Language Descriptionscited by 3
  5. 2026LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Spacecited by 3
  6. 2026Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTScited by 3
  7. 2026Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speechcited by 3
  8. 2026A Geometric Perspective on Composable Emotion Steering in Text-to-Speech Modelscited by 3
  9. 2025UltraVoice: Scaling Fine-Grained Style-Controlled Speech Conversations for Spoken Dialogue Modelscited by 2
  10. 2026Unifying Speech Recognition, Synthesis and Conversion with Autoregressive Transformerscited by 2
  11. 2026UniTalking: A Unified Audio-Video Framework for Talking Portrait Generationcited by 2
  12. 2026Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Modelscited by 2
  13. 2026SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generationcited by 2
  14. 2026TTS-PRISM: A Perceptual Reasoning and Interpretable Speech Model for Fine-Grained Diagnosiscited by 2
  15. 2026Same Words, Different Actions: Paired Turn-Taking Evaluation under Rewritten Dialogue Contextscited by 2
  16. 2026DeepASMR: LLM-Based Zero-Shot ASMR Speech Generation for Anyone of Any Voicecited by 1
  17. 2026A2-LLM: An End-to-end Conversational Audio Avatar Large Language Modelcited by 1
  18. 2026TiCo: Time-Controllable Spoken Dialogue Modelcited by 1
  19. 2026CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbingcited by 1
  20. 2026The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluationcited by 1
  21. 2026VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singingcited by 1
  22. 2026Kinetic-Optimal Scheduling with Moment Correction for Metric-Induced Discrete Flow Matching in Zero-Shot Text-to-Speechcited by 1
  23. 2026AgentSteerTTS: A Multi-Agent Closed-Loop Framework for Composite-Instruction Text-to-Speechcited by 1
  24. 2026PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialectscited by 1
  25. 2026WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modelingcited by 1
  26. 2026An Evaluation Framework for Text-to-Speech Voice Reconstructioncited by 1
  27. 2026HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speechcited by 1
  28. 2026How to Leverage Synthetic Speech for LLM-Based ASR Systems?cited by 1
  29. 2026DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speechcited by 1
  30. 2026WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTScited by 1
  31. 2026VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentationcited by 1
  32. 2026Adding Voice Cloning to Text-to-Audio-Video Models with a Single Zero-Initialised Layercited by 1
  33. 2026HUG-VIS: A Multimodal Benchmark for Human-centered Understanding and Generation in Visual Intelligencecited by 1
  34. 2026AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editingcited by 1
  35. 2026DuplexDrama: A Synthesized Dialogue Dataset with Scenarios, Full-Duplex Behaviors, Expressive Speech, and Sound Eventscited by 1
  36. 2026MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversationcited by 1
  37. 2026Decentralized Autoregressive Generationcited by 0
  38. 2026Cross-Linguistic Persona-Driven Data Synthesis for Robust Multimodal Cognitive Decline Detectioncited by 0

50 most cited of 90 All 90 (JSON)