mlsanityData to 7 Oct 2026

MaskGIT: Masked Generative Image Transformer

arXiv 2202.04200 ↗First posted 8 Feb 2022cs.CVcited by 1,082

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Generative transformers have experienced rapid popularity growth in the computer vision community in synthesizing high-fidelity and high-resolution images. The best generative transformer models so far, however, still treat an image naively as a sequence of tokens, and decode an image sequentially following the raster scan ordering (i.e. line-by-line). We find this strategy neither optimal nor efficient. This paper proposes a novel image synthesis paradigm using a bidirectional transformer decoder, which we term MaskGIT. During training, MaskGIT learns to predict randomly masked tokens by attending to tokens in all directions. At inference time, the model begins with generating all tokens of an image simultaneously, and then refines the image iteratively conditioned on the previous generation. Our experiments demonstrate that MaskGIT significantly outperforms the state-of-the-art transformer model on the ImageNet dataset, and accelerates autoregressive decoding by up to 64x. Besides, we illustrate that MaskGIT can be easily extended to various image editing tasks, such as inpainting, extrapolation, and image manipulation.

Key facts. Cited by 1,082 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 44 of its 61 references that are in the map. First posted on arXiv 8 Feb 2022.

Builds on

44 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale22 Oct

Other references · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  11. 2016Layer Normalizationcited by 5,579
  12. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
31 more
  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  13. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  14. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  15. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  16. 2018Self-Attention Generative Adversarial Networkscited by 809
  17. 2018Image Transformercited by 770
  18. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  19. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  20. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  21. 2017Non-Autoregressive Neural Machine Translationcited by 457
  22. 2021Generating Images with Sparse Representationscited by 262
  23. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 219
  24. 2019Classification Accuracy Score for Conditional Generative Modelscited by 134
  25. 2021High-Fidelity Pluralistic Image Completion with Transformerscited by 134
  26. 2020Contextual Residual Aggregation for Ultra High-Resolution Image Inpaintingcited by 124
  27. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  28. 2021Regularizing Generative Adversarial Networks under Limited Datacited by 105
  29. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  30. 2019Boundless: Generative Adversarial Networks for Image Extensioncited by 62
  31. 2021In&Out : Diverse Image Outpainting via GAN Inversioncited by 55

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 47; 2023: 149; 2024: 222; 2025: 376; 2026: 288.

1,082 papers.

Sort:

By area:

  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  3. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  4. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  5. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  6. 2025Large Language Diffusion Modelscited by 872
  7. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  8. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  9. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  10. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
  11. 2024Genie: Generative Interactive Environmentscited by 706
  12. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
38 more
  1. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  2. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  3. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  4. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  5. 2025Mean Flows for One-step Generative Modelingcited by 524
  6. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  7. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  8. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  9. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  10. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  11. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  12. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
  13. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  14. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 376
  15. 2022MAGVIT: Masked Generative Video Transformercited by 341
  16. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  17. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  18. 2024Discrete Flow Matchingcited by 301
  19. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  20. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  21. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  22. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  23. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  24. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  25. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  26. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizercited by 260
  27. 2023Generating Images with Multimodal Language Modelscited by 259
  28. 2022Visual Prompting via Image Inpaintingcited by 248
  29. 2023A Cookbook of Self-Supervised Learningcited by 248
  30. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  31. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  32. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  33. 2022Towards Robust Blind Face Restoration with Codebook Lookup Transformercited by 230
  34. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  35. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  36. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
  37. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  38. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212

50 most cited of 1,082 All 1,082 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels
  2. 2025Toward Lightweight and Fast Decoders for Diffusion Models in Image and Video Generation
  3. 2023Bridging The Gaps Between Token Pruning and Full Pre-training via Masked Fine-tuning
  4. 2023Morphing Tokens Draw Strong Masked Image Models
  5. 2025Token Transforming: A Unified and Training-Free Token Compression Framework for Vision Transformer Acceleration
  6. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes
  7. 2025UNCAGE: Contrastive Attention Guidance for Masked Generative Transformers in Text-to-Image Generation
  8. 2025Learnings from Scaling Visual Tokenizers for Reconstruction and Generation