mlsanityData to 7 Oct 2026

Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation

arXiv 2406.06525 ↗First posted 10 Jun 2024cs.CVcited by 760

Not on the map. Why?

Cited by 154 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce LlamaGen, a new family of image generation models that apply original ``next-token prediction'' paradigm of large language models to visual generation domain. It is an affirmative answer to whether vanilla autoregressive models, e.g., Llama, without inductive biases on visual signals can achieve state-of-the-art image generation performance if scaling properly. We reexamine design spaces of image tokenizers, scalability properties of image generation models, and their training data quality. The outcome of this exploration consists of: (1) An image tokenizer with downsample ratio of 16, reconstruction quality of 0.94 rFID and codebook usage of 97% on ImageNet benchmark. (2) A series of class-conditional image generation models ranging from 111M to 3.1B parameters, achieving 2.18 FID on ImageNet 256x256 benchmarks, outperforming the popular diffusion models such as LDM, DiT. (3) A text-conditional image generation model with 775M parameters, from two-stage training on LAION-COCO and high aesthetics quality images, demonstrating competitive performance of visual quality and text alignment. (4) We verify the effectiveness of LLM serving frameworks in optimizing the inference speed of image generation models and achieve 326% - 414% speedup. We release all models and codes to facilitate open-source community of visual generation and multimodal foundation models.

Key facts. Cited by 760 papers in the map (data through 7 Oct 2026). Builds on 81 of its 98 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

81 of 98 references.

References · 81

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2022Training language models to follow instructions with human feedbackcited by 15,712
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  11. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  12. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
38 more
  1. 2020Denoising Diffusion Implicit Modelscited by 9,655
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  12. 2017Neural Discrete Representation Learningcited by 4,650
  13. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  14. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  15. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  16. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  17. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  18. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  19. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  20. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  21. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  22. 2023Qwen Technical Reportcited by 3,273
  23. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  24. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  25. 2016Improved Techniques for Training GANscited by 2,938
  26. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  27. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  28. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  29. 2022Emergent Abilities of Large Language Modelscited by 2,483
  30. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  31. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  32. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  33. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  34. 2020GLU Variants Improve Transformercited by 1,724
  35. 2019Root Mean Square Layer Normalizationcited by 1,425
  36. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  37. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  38. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 114; 2025: 403; 2026: 243.

760 papers.

Sort:

By area:

  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  2. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  3. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  4. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  5. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
  6. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  7. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  8. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  9. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  10. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  11. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
  12. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 204
38 more
  1. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  2. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  3. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  4. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  5. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 162
  6. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 162
  7. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 153
  8. 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148
  9. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 148
  10. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 130
  11. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
  12. 2024Randomized Autoregressive Visual Generationcited by 125
  13. 2025Video World Models with Long-term Spatial Memorycited by 124
  14. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
  15. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generationcited by 117
  16. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
  17. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editingcited by 103
  18. 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 102
  19. 2025DDT: Decoupled Diffusion Transformercited by 100
  20. 2024Loong: Generating Minute-level Long Videos with Autoregressive Language Modelscited by 99
  21. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 96
  22. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  23. 2025ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generationcited by 94
  24. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 93
  25. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  26. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 93
  27. 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 91
  28. 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 90
  29. 2024Normalizing Flows are Capable Generative Modelscited by 89
  30. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RLcited by 85
  31. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  32. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 83
  33. 2025Latent Diffusion Model without Variational Autoencodercited by 83
  34. 2024OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Drivingcited by 82
  35. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 82
  36. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  37. 2025OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generationcited by 81
  38. 2024Addressing Representation Collapse in Vector Quantized Models with One Linear Layercited by 77

50 most cited of 760 All 760 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes
  2. 2024Stabilize the Latent Space for Image Autoregressive Modeling: A Unified Perspective
  3. 2023Emage: Non-Autoregressive Text-to-Image Generation
  4. 2023StraIT: Non-autoregressive Generation with Stratified Image Transformer
  5. 2022Progressive Text-to-Image Generation
  6. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  7. 2023A Survey of Diffusion Based Image Generation Models: Issues and Their Solutions
  8. 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens