mlsanityData to 14 Oct 2026

DiTAR+: Dual Optimization for Robust Autoregressive Diffusion Speech Synthesis

arXiv 2609.13909 ↗First posted 12 Sep 2026cs.SD cs.AI eess.AScited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Continuous-latent Autoregressive Diffusion Transformer (AR-DiT) models have demonstrated immense potential in zero-shot speech generation. However, they still suffer from limited decoding stability when synthesizing long utterances or complex linguistic structures. This instability primarily stems from a restricted historical receptive field and an acoustic inertia dependency within the diffusion decoder, which causes the model to ignore semantic conditions. To address these challenges, we propose DiTAR+, a dual-optimization framework. First, we introduce Dilated Context Sampling to expand the macro-level historical receptive field without violating physical temporal continuity, thereby preventing cumulative error propagation. Second, we propose Hierarchical Acoustic Masking to prevent shallow layers from attending to acoustic pre-context, explicitly decoupling semantic alignment from acoustic detail reconstruction. Extensive experiments show that our framework effectively mitigates pronunciation errors and semantic hallucinations, enhances generation robustness on challenging sentences, and maintains exceptionally high speaker similarity throughout the entirety of long-form utterances. On the linguistically challenging ZH-Hard set, DiTAR+ reduces the word error rate from 12.478% to 9.893%, and on extended utterances of 25 to 35 seconds it improves speaker similarity from 0.741 to 0.759 while simultaneously lowering the word error rate from 2.778% to 2.173%, outperforming both discrete-token and pure flow-matching baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 23 of its 24 references that are in the map. First posted on arXiv 12 Sep 2026.

Builds on

23 of 24 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation6 Feb
  2. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models13 Dec
  3. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  4. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models5 Mar
  5. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
  6. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr
  7. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models30 Jan
  8. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan

Other references · 15

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  3. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  4. 2022High Fidelity Neural Audio Compressioncited by 687
  5. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  6. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  7. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  8. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  9. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  10. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  11. 2023Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervisioncited by 169
  12. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
3 more
  1. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  2. 2023SoundStorm: Efficient Parallel Audio Generationcited by 114
  3. 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DiSTAR: Diffusion over a Scalable Token Autoregressive Representation for Speech Generation
  2. 2024Improving Robustness of Diffusion-Based Zero-Shot Speech Synthesis via Stable Formant Generation
  3. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesis
  4. 2025M3-TTS: Multi-modal DiT Alignment & Mel-latent for Zero-shot High-fidelity Speech Synthesis
  5. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  6. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis
  7. 2026Fast Text-to-Audio Generation with One-Step Sampling via Energy-Scoring and Auxiliary Contextual Representation Distillation ↗
  8. 2025CLEAR: Continuous Latent Autoregressive Modeling for High-quality and Low-latency Speech Synthesis