mlsanityData to 7 Oct 2026

Autoregressive Image Generation without Vector Quantization

arXiv 2406.11838 ↗First posted 17 Jun 2024cs.CVcited by 644

Not on the map. Why?

Cited by 197 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Conventional wisdom holds that autoregressive models for image generation are typically accompanied by vector-quantized tokens. We observe that while a discrete-valued space can facilitate representing a categorical distribution, it is not a necessity for autoregressive modeling. In this work, we propose to model the per-token probability distribution using a diffusion procedure, which allows us to apply autoregressive models in a continuous-valued space. Rather than using categorical cross-entropy loss, we define a Diffusion Loss function to model the per-token probability. This approach eliminates the need for discrete-valued tokenizers. We evaluate its effectiveness across a wide range of cases, including standard autoregressive models and generalized masked autoregressive (MAR) variants. By removing vector quantization, our image generator achieves strong results while enjoying the speed advantage of sequence modeling. We hope this work will motivate the use of autoregressive generation in other continuous-valued domains and applications. Code is available at: https://github.com/LTH14/mar.

Key facts. Cited by 644 papers in the map (data through 7 Oct 2026). Builds on 44 of its 56 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

44 of 56 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2022Scalable Diffusion Models with Transformerscited by 5,931
32 more
  1. 2022Classifier-Free Diffusion Guidancecited by 5,717
  2. 2016Layer Normalizationcited by 5,579
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  10. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  11. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  13. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  14. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  15. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  16. 2016Pixel Recurrent Neural Networkscited by 814
  17. 2018Image Transformercited by 770
  18. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  19. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  20. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  21. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  22. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  23. 2022MAGVIT: Masked Generative Video Transformercited by 341
  24. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  25. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizercited by 260
  26. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  27. 2017PixelSNAIL: An Improved Autoregressive Generative Modelcited by 137
  28. 2023DiffiT: Diffusion Vision Transformers for Image Generationcited by 104
  29. 2022UViM: A Unified Modeling Approach for Vision with Learned Guiding Codescited by 77
  30. 2023Diffusion Models as Masked Autoencoderscited by 71
  31. 2013Deep AutoRegressive Networkscited by 60
  32. 2024Denoising Autoregressive Representation Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 83; 2025: 353; 2026: 208.

644 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  2. 2025Mean Flows for One-step Generative Modelingcited by 524
  3. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  4. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  5. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
  6. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  7. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  8. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  9. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  10. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  11. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  12. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
38 more
  1. 2024Deep Compression Autoencoder for Efficient High-Resolution Diffusion Modelscited by 209
  2. 2025Unified Video Action Modelcited by 195
  3. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  4. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 168
  5. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  6. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 148
  7. 2024Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokenscited by 147
  8. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 130
  9. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
  10. 2024Randomized Autoregressive Visual Generationcited by 125
  11. 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
  12. 2025GR-3 Technical Reportcited by 110
  13. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
  14. 2025Inductive Moment Matchingcited by 107
  15. 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 102
  16. 2025Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decodingcited by 101
  17. 2025DDT: Decoupled Diffusion Transformercited by 100
  18. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 96
  19. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 93
  20. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  21. 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformercited by 91
  22. 2024Normalizing Flows are Capable Generative Modelscited by 89
  23. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 85
  24. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RLcited by 85
  25. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 82
  26. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
  27. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  28. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 81
  29. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  30. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 77
  31. 2024Dual Diffusion for Unified Image Generation and Understandingcited by 76
  32. 2025PixNerd: Pixel Neural Field Diffusioncited by 75
  33. 2025Harmonizing Visual Representations for Unified Multimodal Understanding and Generationcited by 74
  34. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 73
  35. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  36. 2024A Bias-Free Training Paradigm for More General AI-generated Image Detectioncited by 67
  37. 2024Autoregressive Speech Synthesis without Vector Quantizationcited by 65
  38. 2024Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decodingcited by 65

50 most cited of 644 All 644 (JSON)