mlsanityData to 14 Oct 2026

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

arXiv 2404.02905 ↗First posted 3 Apr 2024cs.CV cs.AIcited by 1,023

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution prediction", diverging from the standard raster-scan "next-token prediction". This simple, intuitive methodology allows autoregressive (AR) transformers to learn visual distributions fast and generalize well: VAR, for the first time, makes GPT-like AR models surpass diffusion transformers in image generation. On ImageNet 256x256 benchmark, VAR significantly improve AR baseline by improving Frechet inception distance (FID) from 18.65 to 1.73, inception score (IS) from 80.4 to 350.2, with around 20x faster inference speed. It is also empirically verified that VAR outperforms the Diffusion Transformer (DiT) in multiple dimensions including image quality, inference speed, data efficiency, and scalability. Scaling up VAR models exhibits clear power-law scaling laws similar to those observed in LLMs, with linear correlation coefficients near -0.998 as solid evidence. VAR further showcases zero-shot generalization ability in downstream tasks including image in-painting, out-painting, and editing. These results suggest VAR has initially emulated the two important properties of LLMs: Scaling Laws and zero-shot task generalization. We have released all models and codes to promote the exploration of AR/VAR models for visual generation and unified learning.

Key facts. Cited by 1,023 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 89 of its 105 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

89 of 105 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Paths29 May
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 86

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  8. 2023GPT-4 Technical Reportcited by 18,396
  9. 2022Training language models to follow instructions with human feedbackcited by 15,971
  10. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  11. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  12. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
38 more
  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2020Denoising Diffusion Implicit Modelscited by 9,781
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2023Segment Anythingcited by 8,234
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2020Scaling Laws for Neural Language Modelscited by 6,586
  9. 2022Scalable Diffusion Models with Transformerscited by 6,054
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  11. 2022Classifier-Free Diffusion Guidancecited by 5,847
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  13. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  14. 2017Neural Discrete Representation Learningcited by 4,712
  15. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  16. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  17. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  18. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  19. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  20. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  21. 2023Qwen Technical Reportcited by 3,344
  22. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  23. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  24. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  25. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  26. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  27. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  28. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  29. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  30. 2022Visual Prompt Tuningcited by 1,681
  31. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  32. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  33. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  34. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  35. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  36. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  37. 2023PaLM 2 Technical Reportcited by 1,162
  38. 2022MaskGIT: Masked Generative Image Transformercited by 1,099

50 most cited of 86 All 86 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 147; 2025: 504; 2026: 372.

1,023 papers.

Sort:

By area:

  1. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  2. 2025Mean Flows for One-step Generative Modelingcited by 561
  3. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  4. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  5. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 393
  6. 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 366
  7. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  8. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  9. 2025Diffusion Transformers with Representation Autoencoderscited by 295
  10. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 283
  11. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  12. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
38 more
  1. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
  2. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 211
  3. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
  4. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 202
  5. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
  6. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  7. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 155
  8. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 153
  9. 2024Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokenscited by 147
  10. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 145
  11. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 133
  12. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 132
  13. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 132
  14. 2024Randomized Autoregressive Visual Generationcited by 127
  15. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generationcited by 118
  16. 2024Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabulariescited by 118
  17. 2025Inductive Moment Matchingcited by 113
  18. 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 110
  19. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 109
  20. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 102
  21. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  22. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 98
  23. 2025Controllable Video Generation: A Surveycited by 98
  24. 2025ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generationcited by 97
  25. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  26. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 96
  27. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 94
  28. 2024Normalizing Flows are Capable Generative Modelscited by 91
  29. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 89
  30. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 87
  31. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RLcited by 86
  32. 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actionscited by 86
  33. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  34. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Drivingcited by 83
  35. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 83
  36. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 80
  37. 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
  38. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 74

50 most cited of 1,023 All 1,023 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Universal Approximation of Visual Autoregressive Transformers
  2. 2024M-VAR: Decoupled Scale-wise Autoregressive Modeling for High-Quality Image Generation
  3. 2025Visual Autoregressive Modeling for Image Super-Resolution
  4. 2025MADFormer: Mixed Autoregressive and Diffusion Transformers for Continuous Image Generation
  5. 2025Fast Autoregressive Models for Continuous Latent Generation
  6. 2025NSARM: Next-Scale Autoregressive Modeling for Robust Real-World Image Super-Resolution