mlsanityData to 7 Oct 2026

Autoregressive Image Generation using Residual Quantization

arXiv 2203.01941 ↗First posted 3 Mar 2022cs.CV cs.LGcited by 706

Not on the map. Why?

Cited by 95 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

For autoregressive (AR) modeling of high-resolution images, vector quantization (VQ) represents an image as a sequence of discrete codes. A short sequence length is important for an AR model to reduce its computational costs to consider long-range interactions of codes. However, we postulate that previous VQ cannot shorten the code sequence and generate high-fidelity images together in terms of the rate-distortion trade-off. In this study, we propose the two-stage framework, which consists of Residual-Quantized VAE (RQ-VAE) and RQ-Transformer, to effectively generate high-resolution images. Given a fixed codebook size, RQ-VAE can precisely approximate a feature map of an image and represent the image as a stacked map of discrete codes. Then, RQ-Transformer learns to predict the quantized feature vector at the next position by predicting the next stack of codes. Thanks to the precise approximation of RQ-VAE, we can represent a 256×256 image as 8×8 resolution of the feature map, and RQ-Transformer can efficiently reduce the computational costs. Consequently, our framework outperforms the existing AR models on various benchmarks of unconditional and conditional image generation. Our approach also has a significantly faster sampling speed than previous AR models to generate high-quality images.

Key facts. Cited by 706 papers in the map (data through 7 Oct 2026). Builds on 33 of its 54 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

33 of 54 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  8. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2017Neural Discrete Representation Learningcited by 4,650
  12. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
21 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  9. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  10. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  11. 2016Pixel Recurrent Neural Networkscited by 814
  12. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  13. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  14. 2020Jukebox: A Generative Model for Musiccited by 535
  15. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  16. 2021Generating Images with Sparse Representationscited by 262
  17. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  18. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  19. 2014Stacked Quantizers for Compositional Vector Compressioncited by 30
  20. 2017Performance Guaranteed Network Acceleration via High-Order Residual Quantizationcited by 24
  21. 2017Regularized Residual Quantization: a multi-layer sparse dictionary learning approachcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 13; 2023: 59; 2024: 143; 2025: 259; 2026: 232.

706 papers.

Sort:

By area:

  1. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  2. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  3. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  4. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  5. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  6. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  7. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  8. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  9. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  10. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  11. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  12. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
38 more
  1. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  2. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  3. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  4. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  5. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  6. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  7. 2023StyleDrop: Text-to-Image Generation in Any Stylecited by 197
  8. 2023Recommender Systems with Generative Retrievalcited by 191
  9. 2023Flow Matching in Latent Spacecited by 155
  10. 2024Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generationcited by 153
  11. 2025UniTok: A Unified Tokenizer for Visual Generation and Understandingcited by 148
  12. 2024HART: Efficient Visual Generation with Hybrid Autoregressive Transformercited by 128
  13. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125
  14. 2024Randomized Autoregressive Visual Generationcited by 125
  15. 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
  16. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generationcited by 117
  17. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  18. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  19. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  20. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  21. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  22. 2023DiffiT: Diffusion Vision Transformers for Image Generationcited by 104
  23. 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 102
  24. 2024MaskBit: Embedding-free Image Generation via Bit Tokenscited by 96
  25. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  26. 2023Return of Unconditional Generation: A Self-supervised Representation Generation Methodcited by 95
  27. 2024Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animationcited by 95
  28. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  29. 2024ImageFolder: Autoregressive Image Generation with Folded Tokenscited by 93
  30. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  31. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 86
  32. 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actionscited by 85
  33. 2024RandAR: Decoder-only Autoregressive Visual Generation in Random Orderscited by 82
  34. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
  35. 2023Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networkscited by 80
  36. 2024Addressing Representation Collapse in Vector Quantized Models with One Linear Layercited by 77
  37. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 77
  38. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71

50 most cited of 706 All 706 (JSON)