mlsanityData to 7 Oct 2026

DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation

arXiv 2410.08159 ↗First posted 10 Oct 2024cs.CV cs.LGcited by 32

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Diffusion models have become the dominant approach for visual generation. They are trained by denoising a Markovian process which gradually adds noise to the input. We argue that the Markovian property limits the model's ability to fully utilize the generation trajectory, leading to inefficiencies during training and inference. In this paper, we propose DART, a transformer-based model that unifies autoregressive (AR) and diffusion within a non-Markovian framework. DART iteratively denoises image patches spatially and spectrally using an AR model that has the same architecture as standard language models. DART does not rely on image quantization, which enables more effective image modeling while maintaining flexibility. Furthermore, DART seamlessly trains with both text and image data in a unified model. Our approach demonstrates competitive performance on class-conditioned and text-to-image generation tasks, offering a scalable, efficient alternative to traditional diffusion models. Through this unified framework, DART sets a new benchmark for scalable, high-quality image synthesis.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 47 of its 55 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

47 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024OmniGen: Unified Image Generation17 Sep
  2. 2024Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling31 May
  3. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction3 Apr
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May

Other references · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2024The Llama 3 Herd of Modelscited by 14,192
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2022Scalable Diffusion Models with Transformerscited by 5,931
31 more
  1. 2022Classifier-Free Diffusion Guidancecited by 5,717
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  10. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  11. 2020GLU Variants Improve Transformercited by 1,724
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  13. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  14. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  15. 2021Variational Diffusion Modelscited by 1,145
  16. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  17. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  18. 2016Pixel Recurrent Neural Networkscited by 814
  19. 2018Image Transformercited by 770
  20. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  21. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  22. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  23. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  24. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  25. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  26. 2023Diffusion Models Without Attentioncited by 75
  27. 2023Matryoshka Diffusion Modelscited by 70
  28. 2024MonoFormer: One Transformer for Both Diffusion and Autoregressioncited by 45
  29. 2022f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformationcited by 32
  30. 2023Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modelingcited by 13
  31. 2024Computational Tradeoffs in Image Synthesis: Diffusion, Masked-Token, and Next-Token Predictioncited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 19; 2026: 3.

32 papers.

Sort:

By area:

  1. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
  2. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  3. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 76
  4. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  5. 2024ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformercited by 45
  6. 2024Switti: Designing Scale-Wise Transformers for Text-to-Image Synthesiscited by 33
  7. 2024Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficientcited by 32
  8. 2024Causal Diffusion Transformers for Generative Modelingcited by 31
  9. 2025STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesiscited by 31
  10. 2024CARP: Visuomotor Policy Learning via Coarse-to-Fine Autoregressive Predictioncited by 26
  11. 2024Next Patch Prediction for Autoregressive Visual Generationcited by 25
  12. 2025FlexVAR: Flexible Visual Autoregressive Modeling without Residual Predictioncited by 22
20 more
  1. 2025Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compressioncited by 22
  2. 2025The Power of Context: How Multimodality Improves Image Super-Resolutioncited by 15
  3. 2024Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspectivecited by 10
  4. 2024Text-to-Image Synthesis: A Decade Surveycited by 9
  5. 2025Autoregressive Distillation of Diffusion Transformerscited by 9
  6. 2025MVAR: Visual Autoregressive Modeling with Scale and Spatial Markovian Conditioningcited by 8
  7. 2025ARFlow: Autoregressive Flow with Hybrid Linear Attentioncited by 7
  8. 2025SRDiffusion: Accelerate Video Diffusion Inference via Sketching-Rendering Cooperationcited by 7
  9. 2025Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generationcited by 7
  10. 2025SimFlow: Simplified and End-to-End Training of Latent Normalizing Flowscited by 6
  11. 2026EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Modelscited by 6
  12. 2025Next Visual Granularity Generationcited by 5
  13. 2025GarmentX: Autoregressive Parametric Representations for High-Fidelity 3D Garment Generationcited by 3
  14. 2025From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generationcited by 3
  15. 2026Normalizing Trajectory Modelscited by 2
  16. 2025LSRS: Latent Scale Rejection Sampling for Visual Autoregressive Modelingcited by 1
  17. 2025CAM-Seg: A Continuous-valued Embedding Approach for Semantic Image Generationcited by 0
  18. 2025D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokenscited by 0
  19. 2025Rethinking Training Dynamics in Scale-wise Autoregressive Generationcited by 0
  20. 2026XYZFlow:Scaling Multi dimensional Shortcut Flows for Efficient Generative Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Emage: Non-Autoregressive Text-to-Image Generation
  2. 2025MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation
  3. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  4. 2026DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers
  5. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
  6. 2024E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling
  7. 2025Marrying Autoregressive Transformer and Diffusion with Multi-Reference Autoregression
  8. 2026Efficient Text-to-Image Generation: An Adaptive Step Schedule Controller for Diffusion Models